This post is dedicated to two people who are frequent commenters on SBM, Stephen S. Rodrigues and Peter Moran. Rodrigues is an MD/acupuncturist who tries to persuade us that acupuncture is effective. Moran is a retired surgeon who objects to insulting language and thinks more can be accomplished by trying to better understand why people turn to CAM and by explaining the facts and reasons politely and dispassionately. He has claimed that he “could probably help [Rodrigues] understand better why his views are not having much impact.”
I recently wrote about supplements for age-related macular degeneration (AMD). There is evidence that the supplement mixture tested in the AREDS trial slows the progression of moderate to advanced disease. That is based on a good scientific study, although the study has not been replicated and there is reason to interpret its results with caution. Dr. Rodrigues commented with a link to a website advertising the Santa Fe acupuncture protocol, saying that he uses the method in his office and it helps some of his patients with AMD. The website claims that the Santa Fe acupuncture protocol will reverse vision loss from macular degeneration in 4 days or your money back. That is a bold claim. I will try to explain, as politely as possible, why I reject the claim, and why the evidence for it is unacceptable.
Two kinds of AMD: dry and wet
90% of AMD patients have dry AMD, where there is a thinning and breakdown of the retinal epithelial cells that support the photoreceptor cells (the rods and cones). It is asymptomatic in the early stages; then, as the disease progresses, visual acuity is lost and the unsupported photoreceptor cells die off. There is no medical or surgical treatment for dry AMD.
The other 10% have wet AMD, where blood vessels proliferate, leak, interfere with vision, and eventually destroy photoreceptor cells. Conventional treatment can’t cure AMD; it can slow progression of the disease but until the recent introduction of anti-VEGF drugs it seldom improved visual acuity. Treatments include injecting potentially dangerous medications directly into the eye, intravenous injection of drugs with retinal photoactivation, photocoagulation with lasers (which can cause immediate reduction in visual acuity of three lines on the eye chart), and surgical removal of retinal lesions.
Why people turn to CAM
I fully understand the attraction of the Santa Fe protocol. People with AMD are desperate; they are going blind, and conventional treatments offer them little or no hope and are frankly scary. When they are told that an alternative treatment can reverse the damage that has already occurred, and when they are told the treatment is supported by testimonials, scientific studies, and a money-back guarantee, how could they not grasp at the only available straw of hope? Hope makes people feel better; but raising false hopes only to have them collapse after a substantial investment of time and money would ultimately make them feel worse. Does this protocol offer true hope or false hope? Let’s examine the evidence.
I can understand the attraction for acupuncturists, too. They want to believe they can help patients with a condition that can’t be treated any other way. Their patients report improvement and thank them profusely. Those who don’t improve are not likely to come back, so practitioners don’t see their successes in perspective with their failures. Confirmation bias takes over. Practitioners are gratified and reinforced. That kind of personal experience is very powerful but can be misleading. As Mark Crislip says, the three most dangerous words in medicine are “In my experience.” Humans regularly misperceive and misinterpret the meaning of their experiences, and that’s why we have to rely on the scientific method to correct our errors.
What is the Santa Fe protocol?
Dr. Alston C. Lundgren is a family physician turned acupuncturist. His website claims that his Santa Fe protocol is the only documented treatment to reverse vision loss in cases of both wet and dry AMD.
Apparently there are several versions of the protocol, as he added treatments through the years. On his website he describes three components:
- Small steel studs are inserted in regions of the ear whose nerve endings have been shown to correspond with specific areas of the brain. By stimulating these nerves, cortisone and oxygen-rich blood are induced in the region of the retina.
- Needle electrodes are inserted in the fat around the eyeball and are stimulated with a mild electrical current. These electrodes stimulate the retina and surrounding tissues and probably increase blood flow to the retina.
- In Spring 2007, a 3rd component was added – electrically stimulating the scalp over the visual cortex which occupies a considerable portion of the brain.
He says he has performed over 10,000 treatments on 1,500 patients and has improved the vision of 85% of them. He states that black eyes are a very common complication, so he routinely uses ice packs. He charges $250 per treatment, and typically administers five treatments on five successive days.
In one of his links, he describes his protocol as combining five types of acupuncture:
- German Ear Acupuncture to indirectly stimulate the optic nerve, the retina, and production of cortisol. (David Alimi, MD, Professor of Neurology and Ear Acupuncture at the University of Paris Medical School has demonstrated the one-to-one correspondence between points on the ear and areas of the brain.
- Canadian Neuroanatomic Acupuncture with needle/electrodes surrounding the orbit which may increase blood circulation or directly affect the retina itself when electrically stimulated.
- Chinese scalp acupuncture – electrical stimulation over the visual cortex of the brain, hence reversing the damage of a stroke there.
- Japanese scalp acupuncture at points identified as stimulating the optic nerve and probably other areas of the brain involved in vision.
- French Energetics acupuncture to stimulate classic Chinese points affecting the eye, increase parasympathetic stimulus, and add energy to the patient for self-healing.
Lundgren is the only one who has published data on his method. He has no articles listed in PubMed. His website provides references, but they are all either articles he has written for an acupuncture journal that is not listed in PubMed (I couldn’t find it in any list of medical journal impact factors either) or PowerPoints he has presented at conferences. They all cover essentially the same material at different points in his journey, reporting data from his patients.
His published study
The most recent published article is this one published in the journal Medical Acupuncture in 2005. It reports a 69% improvement in vision. I will concentrate on the evidence for the Santa Fe protocol as presented in that study.
It reports a case series of 108 consecutive patients with ophthalmologist-diagnosed AMD who were treated by him at his clinic in New Mexico. 32% had wet AMD; 50% had dry AMD; 18% didn’t know which type they had. The patients gave informed consent, but there is no mention of oversight by an IRB. Treatment consisted of three acupuncture techniques:
Auricular acupuncture to indirectly stimulate appropriate parts of the brain
Neuro-anatomical acupuncture to directly stimulate the retina and periorbital tissue
French Energetic Liver cerebral circulation stimulation to enhance eye function
For auricular acupuncture he says he does not clean the surface of the ear before inserting needles unless there is gross contamination. He leaves the needles in the ear until they fall out, anywhere from 2 days to more than a month later. He installs a gold semi-permanent needle (a stud?) for chronic stimulation of the most electrically active point. He applies electrical stimulation to the needles in techniques 2 and 3, but I couldn’t tell from his write-up whether he also uses electricity on the ear. He reports elsewhere that he treats daily for 5 days; but in this study he used weekly treatments, saying patients did not benefit from more than 3 treatments per week, and claiming that gaps of up to 4 months between treatments had no effect on the degree of improvement achieved. Each treatment session lasted 25-35 minutes. Patients were treated until they either dropped out or showed no further gains in 2 consecutive vision tests. He does not say how many patients dropped out. He says visual acuity improvements “did not seem to deteriorate for a year-and-a-half” but he provides no data. He describes other subjective improvements reported by patients: color vision seemed to improve (said to have been confirmed by testing in “several” patients), less contrast required for reading, improved ability to see in dim light or drive at night, straight objects no longer had wavy edges, scotomas shrank and disappeared, the “film over vision” or “fog” disappeared.
There were minor complications: ecchymosis in 10% of treatments, and a few infections around indwelling ear studs.
He is aware of some of the limitations of his study. He acknowledges that further research is needed to determine that the results are due to the acupuncture protocol and are not skewed by patients’ increased experience in taking the tests, their desire to please the investigator by performing well, or the Hawthorne effect from simply being part of a study. He wonders if the improvement might be due to effects on eye muscle rather than on the retina. He recommends further research to optimize the protocol.
Why I reject this study
I will refrain from commenting on the rationale of using acupuncture for AMD and the possible underlying mechanisms. I’ll just say that the prior plausibility is very low. Of course, prior probability would have to be re-assessed if acupuncture could clearly be shown to work better than a placebo. Does this study constitute evidence that it works? I don’t think so.
I’ll try to explain in simple terms why I reject this study as evidence.
There was no control group. When you do almost anything, patients seem to improve; sometimes they improve if you do nothing, just from fluctuations in the natural course of illness. With any kind of intervention, they will consistently get better results than if they were just assigned to a waiting list. We can’t conclude a treatment is effective until we can be certain that patients wouldn’t have improved to the same degree without the treatment. Lundgren himself has pointed out several factors that could have invalidated the results and that could have been ruled out by using a control group (increased test-taking experience, desire to please the doctor, the Hawthorne effect).
Selection bias could have influenced the results. Subjects were not a representative sample of patients with AMD; they were self-selected and were a sample of convenience (consecutive patients in his practice). It was not representative of the general population, since it included far more patients with wet AMD than expected. He is known for using acupuncture, so those who sought him out probably had a prior belief in acupuncture, or at least were favorably inclined. They were paying him out of pocket and had a vested interest in the treatment. I don’t know if that would make a difference, but good science tries to control for all such possible confounders.
There was no blinding. The researcher himself treated all the patients. When testing visual acuity, he knew if it was a pre-treatment or post-treatment test, so he may well have been consciously or unconsciously biased in the way he conducted the test. Anyone who tests eyes can tell you encouragement improves performance. When I read the lowest line I can read on the eye chart, if the examiner encourages me to try the next line, I can usually read most of that line correctly too, although it feels to me like I’m just guessing.
The study design and reporting were sloppy. Treatments were given at varying intervals, techniques from different schools of acupuncture were combined in an apparently arbitrary fashion and sometimes were chosen because of anecdotal reports for other eye conditions. The number of dropouts was not reported, and he said some subjects dropped out because they weren’t improving, which would skew results based only on those who remained. It was scattershot: there were so many interventions at so many acupoints that there is no way to determine which were important.
The protocol is inconsistent with his descriptions elsewhere. I am left in confusion as to exactly what “The Santa Fe protocol” is. He keeps changing it. Is it eye/ear, eye/ear/scalp or eye/ear/elsewhere-in-body or eye/ear/two different kinds of scalp acupuncture? Is it daily for 5 days or weekly or at even longer irregular intervals?
Inadequate followup. He says the duration of improvement was “substantial” but provides no supporting data. He doesn’t report trying to follow up on dropouts.
No ophthalmologist was involved. Patients reported having been diagnosed with AMD, but we have no details; many of them didn’t even know whether they had the wet or dry type. A retinal specialist would have been able to examine the retina with an ophthalmoscope and dilation and do tests like fluorescein angiography, visual field testing for scotomas, and optical coherence tomography to determine whether the improved visual acuity was a reflection of improvement of the AMD or a spurious finding.
The results were inconsistent with his reports elsewhere: 69% improvement vs. 85-93%. It appears that efficacy was lower when more acupuncture techniques were added.
No IRB approval. It is unethical to do human experimentation without prior approval by an institutional review board to protect the rights of human subjects.
Publication in an obscure journal. It may be peer-reviewed, but only by other acupuncturists who would be inherently biased in favor of their discipline. I don’t know if he tried submitting it to any respected mainstream medical journals, but I am sure it would not meet their standards without substantial revision.
More. I could go on, but that’s enough to make my point.
In my opinion, this study is not good science and we cannot trust its results. It gives the appearance of having been designed not to rigorously investigate whether acupuncture might be effective for AMD, but to justify and promote the researcher’s own practice and to provide support for other acupuncturists who want to use it for that purpose.
What it could have been
This could have been good science. He could have recruited a retinal specialist, blinded the testing of visual acuity, documented detectable changes in the status of the disease itself, used an acceptable control group (perhaps stimulating the “wrong” acupoints, or randomizing with patients under routine care by a retinal specialist). He could have gotten IRB approval and been published in a PubMed-listed mainstream medical journal so others could have checked his work.
Conclusion
Studies like this are legion. We know that half of published research findings are false. When early positive studies, especially uncontrolled studies, are followed by more rigorous studies with an appropriate control group, the positive findings all too often evaporate. The only research on the Santa Fe protocol is by one acupuncturist; his work has not been replicated, and there has been no independent confirmation of any effect of acupuncture in AMD. The only other study I could find through a PubMed search for “acupuncture” and “AMD” is in Chinese and it is not relevant to the Santa Fe method: it compares a single method, traditional Chinese acupuncture, to treatment with medication. I don’t read Chinese, but as far as I could tell from the abstract, it doesn’t even seem to have any acupoints in common with Lundgren’s study.
It is highly unlikely that acupuncture or any other treatment could restore visual function in a part of the retina where the rod and cone cells have died; just as unlikely as the re-growth of an amputated limb. That would be extraordinary and would require extraordinary evidence, far stronger than the evidence of this study. I can’t rule out the possibility that acupuncture might improve vision in the undamaged areas of the retina by some mechanism, but in view of the entire body of evidence from acupuncture studies, that seems unlikely.
OK. I was as polite as I could possibly be. I was nice. I expressed understanding and sympathy for patients and providers. I didn’t insult or disparage the researcher. I didn’t say “It doesn’t work.” I didn’t advise people not to try the treatment. I looked at the evidence, found it wanting, and explained why. Does my post measure up to Dr. Moran’s ideals of civility? I think so. Will it help Dr. Rodrigues understand better why his views are not having much impact? I suspect it will not.
I welcome comments on the persuasiveness of this kind of post in comparison to posts with other approaches.
This article was originally published in the Science- Based Medicine Blog.
