I used to have a cat. That cat was pretty mean. He hated other people and animals. He messed up my house. I’ll never have another cat because they don’t make good house pets.
The little story above is an anecdote. It has characters, reflects a real-life experience in a narrative form and is intended to provide you with “facts”, an opinion and my reasoning for the conclusion I’ve made there at the end.
Did it convince you? Perhaps – if you are open to the idea that cats are bad pets. Is it generalizable to the entire population of people considering pets? No. It’s simply one person’s experience with a cat.
In telling this story, I did something called “sharpening” and “leveling” (see Gilovich). Sharpening is when I emphasize the points I hope the listener will take away from the story. Leveling means I’ve deemphasized or discarded certain details that don’t enhance or support the message I wish to relate. I’ve used some powerful words – “mean”, “hated”, “messed up” “never” – to sharpen my message. These words tend to evoke images which enhance your memory of the story. You’ve now attached some feelings and visual interpretation to the message.
The need to keep the listener engaged and to drive home my message allowed me to engage in some distortion of the story. Here is where anecdotes not only become useless as evidence about the world, they can be misleading.
Imagine this story:
My friend took her child for a checkup where he received his routine vaccinations. Soon after, the child began acting ill. He stopped talking and was less active. She’s convinced something in that shot caused damage to the child.
The same problems arise in this second story. What is being emphasized? More importantly, what is being left out? Is the conclusion justified? In both the above stories, the speaker has condensed a series of uncontrolled events with complicated details into a very short, biased exposition. [Dr. Steve Novella has discussed anecdotes in two great blog posts – here and here.]
While the observations may be perfectly valid, the stated conclusions are not justified in either case.
First, we have not considered all possible interpretations. The speaker has settled on one that is personally comfortable – emotionally or cognitively.
Second, the conclusion is based on one episode, one sample point. We aren’t impressed by small sample sizes in scientific research. Why should we be impressed by one story? Many skeptics aren’t impressed. Anecdotal evidence has a low value for science in opposition to its high value for the public. This value gap creates misunderstandings and communication frustration between scientists and non-scientists, experts and laypersons.
This fundamental difference in the way we consider anecdotes is a gaping chasm between skeptics and believers.
How would you react if on the listening end of the above two stories? Generally, most people are non-confrontational. It is not socially acceptable to challenge the veracity of the speaker’s story. The act of relating an anecdote, with emotional content and dramatic presentation, mutes criticism. Only those with an understanding of critical thinking might question hearsay (out loud or silently) and seek further evidence beyond this single episode. Too many people embrace the info, consider it knowledge and now use that knowledge when making a decision. They may also pass it on.
The simple act of putting a story out there for a listener results in instant credibility (someone told me so) and availability (‘I’ve heard this story…’) of that viewpoint. The availability heuristic is a mental shortcut where we use information that is most readily accessible to us (in our memories) in order to make our decisions. Therefore, if we just heard one story that stuck in our heads or if the media bombards us with examples, we recall these examples readily and they influence our thinking.
Why anecdotes are problematic
Consider all the practices that go into a valid scientific study – large, representative sample sizes, systematic observations, control groups, blinded procedures, attempts to eliminate confounding variables. None of these are considerations in anecdotal evidence.
We have plenty of results that show just how unreliable anecdotal evidence is compared with rigorous studies. That’s why scientific procedures had to be used in trials – because anecdotes aren’t good enough to provide precise, reliable information.
Scientific procedures are designed to remove the biases, subjectivity and emotion from the results. We aim to get an answer that is, to the best of our ability, not what we want it to be. We may get an answer that is not emotionally or cognitive comfortable to all. But that doesn’t make it less true.
I’d like simple medicines to work – Vitamin C for a cold, herbs for depression, a dietary supplement to boost my memory. So, if I try them and I believe they might work and my condition improves (by whatever means), then I will conclude it works. It’s naïve to think my simple “experiment” was conclusive. I’ve made a hasty generalization instead. Yet, it sits well with my worldview and I’m going to hold onto it.
There must be something to all these stories
Oh, there are thousands of sightings of Bigfoot and UFOs throughout history. You can’t build a strong case with little bits and shards of tainted truths. Science professionals will not be impressed by the “many cups of weak coffee poured together to create one strong cup”. One high-quality piece of evidence will do nicely. That smoking gun hasn’t been presented.
Those who believe in a paranormal explanation for these events see science rejecting anecdotes and take it as a personal affront. They interpret scientific scoffing as accusations of lying, or that the observer was under the influence of some drug or just stupid. Or, they see science as denialistic – unwilling to give up a beloved worldview. In the extreme case, they see science as a covert conspiracy to hide an uncomfortable truth from the world.
None of the above is accurate. Sure, academics and skeptics may scoff at such stories because decades of searching with modern technology failed to pin them down. I have a obligation to be critical of one story that, if true, would overturn a long-supported, well-established body of knowledge.
I think there is more to see than us vs them, the skeptics vs believers. I’m interested in why people say they’ve seen things or had experiences. What happens in the space between what they experience and how they interpret that experience? Why do they invoke risky paranormal or scientifically unsupported conclusions?
I don’t have an emotional or cognitively satisfying answer to that.
References
- Baesler, E.J., J.K. Burgoon (1994) “The Temporal Effects of Story and Statistical Evidence on Belief Change”, Communication Research 21(5) pp. 582-602.
- Enkin, M.W. and A.R. Jadad (1998), Using Anecdotal information in evidence-based health care: Heresy or necessity? Annals of Oncology 9 p. 963-966.
- Gilovich, T. (1991) How we know what isn’t so. The Free Press. 216p.
- Novella, S. “The Role of Anecdotes in Science-based Medicine”, Science-Based Medicine blog, January 30, 2008 http://www.sciencebasedmedicine.org/?p=33
- Novella, S. “No Love for Anecdotes”, Neurologica blog, March 8, 2007.
http://www.theness.com/neurologicablog/?p=174
Those who readily accept weak evidence are often awash in various forms of fallacious reasoning.
Perhaps an oblique strategy of first dealing with those logic errors might prove more fruitful?
I’m not sure how to do that. I wish someone would write a book on how to address really awful arguments on the lines of the Creationists versions of how to piss off an evolutionist. I have not figured out how to tactfully tell someone their reasoning is bogus.
Agreed. Any challenge, be it based on logic or evidence, may be seen as a threat, causing one to raise their defenses.
As another idea, many skeptics (including myself) were once believers who readily accepted anecdotes as evidence. What brought us across the threshold into seeing science not as denialistic, but rather the best tool we have for understanding the world?
For me, I think it was the casual evidence. (See the post on “most convincing evidence”) I was open to listening to an alternate view and it stuck. There are those who are not open at all. As we all realize, nothing is going to convince them because, well, it’s a conviction. So, I suppose we just keep making our case the best and most vivid way we can.