Quality of evidence
A frequent complaint from the fringe is that the scientific community is dismissive of the actual evidence for their extraordinary claim. Proponents of psi or UFOs will cite reams of evidence. The scientific community’s standard response is “Extraordinary claims require extraordinary evidence”. That is, the evidence must be high quality, obtained through rigorous testing, with all other explanations accounted for. Overly precautionary experiments are required to support theories that seek to overturn established knowledge. Rampant fraud and gullibility exposed in some fields, especially psi research, require that extra safeguards be used in this research. This is not a double standard because, as Gardner notes for conventional studies, “gerbils don’t cheat” [1]. But humans do, sometimes inadvertently.
Many poorly tested pseudosciences have a characteristic over-reliance on anecdotal evidence. Similar to anecdotes, myths and legends will be used to support a claim. This type of evidence can not be confirmed and is subject to mistakes and misinterpretation. Science considers anecdotal evidence very weak because of the degree of subjectiveness.
In the public view, anecdotal evidence is very persuasive. It is a primary means by which we communicate ideas to each other. To deny an eyewitness account suggests that the researcher considers the eyewitness a lier, under the influence of some altering substance or mentally unstable.
Collected modern and historical anecdotes, bits of circumstantial and questionable evidence, and non-replicated results form an impressive body of evidence to the nonscientist. Even the scientist may feel that there “must be something to it”. The important attention to quality is lost. Lots of weak evidence does not collectively make strong evidence. In an analogy, skeptical investigator Ben Radford equates this to trying to make a cup of strong coffee out of a lot of weak brew [2]. The scientific community is unimpressed and, quite justifiably, turns away.
————
[1] Gardner, M. (1981). Science Good Bad & Bogus, Prometheus Books.
[2] Radford, B. (2009). Personal communication, April 27, 2009.