When I write about a treatment that has been inadequately tested (or that has never been tested or has been tested and shown not to work), someone always asks if I have tried it myself. Apparently, they believe it really works, and if only I would try it for myself, I would believe too. If it worked for me, I would recommend it for everyone. As if my personal anecdote would mean anything!
The underlying assumption seems to be that no one should form any opinion about anything they have not experienced for themselves. That’s totally ridiculous. By that kind of reasoning, I would have to jump out of an airplane without a parachute to find out if the fall would kill me. I would have to take strychnine before I could say it was poisonous. Babies could only be delivered by women who have had babies themselves, ruling out obstetricians and midwives who are male or are childless women. Surgeons should have experienced breast cancer for themselves before they are truly qualified to perform a mastectomy. I shouldn’t say anything about the acupuncture evidence unless I have let someone stick needles in me. No, that’s not how it works.
Early in human history, we had only two ways of learning about the world: our own experience and what others told us about their experiences. But today we have a better way: science—a reliable, systematic method of investigating to find out truths about the world so we no longer have to rely on unreliable personal experience and hearsay. If a treatment hasn’t been scientifically tested, we can’t say that it doesn’t work, but we can say that we don’t know whether it works or not, and we can withhold judgment until proper testing is done. There are too many of those treatments and no reliable method for choosing among them.
Let’s say I tried a treatment for myself. Three things could happen: I could get better, worse, or stay the same, but that wouldn’t tell me anything about the efficacy of the treatment. There was a cartoon in Skeptic magazine years ago about the Blue Dot Cure. The practitioner put a blue dot on the patient’s forehead. If he got better, it was because the blue dot worked; if he got worse, it was because he should have gotten the blue dot sooner; if he stayed the same, it meant the blue dot had kept the illness from getting worse.
“I took a pill; I got better. Therefore, the pill made me better.” That sounds so convincing until you realize that it is the same post hoc ergo propter hoc logical fallacy that reasons that because the sun comes up after the rooster crows, the crowing must be what makes the sun come up.
“It worked for me!” No, you can’t possibly know that. All you can know is that you felt better. In a placebo-controlled asthma experiment, patients reported they could breathe significantly better after the placebos, but that was only their subjective perception. Objective lung function tests showed no improvement. The only way to tell if a treatment works is to show that the users improve more than a control group that gets no treatment or a placebo treatment. And the treatment and control subjects must be as similar as possible to rule out confounders (for instance, the treatment’s effects might be different for smokers, for certain age groups, for diabetics, or for patients taking certain medications). And of course, there can be other confounders that no one has thought of, so no single experiment is definitive.
If you try a treatment for yourself, it may mislead you into thinking it worked when it didn’t. Or it may mislead you into thinking it didn’t work when it usually does; maybe you didn’t use the right preparation or the right dosage, maybe you reacted idiosyncratically to something that works for most people, or maybe you were inadvertently doing something that counteracted the effect. If it didn’t work, it’s always possible that it would work if you tried it again or that it works for most people. So trying it for yourself gives you no valuable information.
Other Possible Reasons for Improvement
If you try a treatment and get better, it’s reasonable to keep using it, but it’s no basis for concluding that it works. There are all kinds of other reasons a person using a treatment might improve even if the treatment is bogus. In the first place, symptoms naturally fluctuate. Patients with arthritis have good days and bad days. If they use a treatment when the symptoms are bad, those bad days will always be followed by better days, so they can’t assume the treatment was responsible. They may reinforce their belief by continuing to use it and seeing improvement whenever the symptoms are bad. Or conditioning may kick in.
A great many diseases are self-limiting. A cold will usually resolve in a week or so whether you treat it or not. Up to 42 percent of uncomplicated urinary tract infections (UTIs) clear on their own. Some people survived pneumonia and other serious infections before we had antibiotics. There are more spontaneous remissions than most people realize, even in cases of advanced metastatic cancer. Diagnoses and prognoses can be wrong. Placebo effects can make patients feel better without improving the outcome of the illness. Psychological and social effects can convince patients they are better when they are not; they want to feel better and they want to get better to please that nice doctor who is trying to help them. Temporary mood improvement may be confused with cure. Memory is fallible. A questionable treatment may get the credit for a cure that was actually due to the surgery or chemotherapy they also got. People are naturally more impressed by stories than by studies, and every snake oil salesman and charlatan can produce impressive testimonials from their customers. And then there’s Aunt Tillie. “Science says homeopathy doesn’t work, but I know it does because my Aunt Tillie swears it cured her.”
Individuals can never know for sure if a treatment cured them, even if it has been proven effective. The studies are done on groups of people, and no treatment is 100 percent effective for everyone. Science can’t yet tell us whether a treatment was responsible for the improvement of an individual. We can’t know for sure that they wouldn’t have improved anyway or that they didn’t improve despite the treatment.
Trying Things for Personal Taste
There is one place where trying something makes sense, but it involves personal taste rather than scientific truth. How can you know you don’t like a food if you haven’t tasted it? When my brother was around twelve, he had never tried pizza but was convinced he wouldn’t like it. When the family ate dinner at a popular pizza restaurant, he was finally persuaded to try a slice. He loved it, and it promptly became one of his favorite foods. And remember Tigger in Winnie the Pooh? He was convinced that thistles were what Tiggers liked best—until he tried them. He continued to try different foods and discovered he didn’t like any of them.
So if someone wants you to try a new food (or music or activity or anything else where tastes differ) because they think you might like it, feel free to try it. But the next time someone urges you to try a questionable medical treatment for yourself as a way to find out if it works, don’t. Just don’t.
This article was originally published as a SkepDoc’s Corner column on the CSI website
