ChiroNexus recently listed the top 10 chiropractic studies of 2013. In my experience, chiropractic studies tend to be of poor quality. A media report says “study shows chiropractic works for X,” and when I look for the study it turns out to be a single case report or an uncontrolled study. When Simon Singh was sued by the British Chiropractic Association for saying chiropractic treatment for certain childhood ailments was bogus, the BCA responded with a list of 29 studies they said provided evidence for their claims. Steven Novella showed that out of 29 studies on the list, only 17 actually constituted evidence for 4 clinical claims, and those 17 were poor quality, cherry-picked, and too weak to support the claims. I have a copy of a chiropractic textbook entitled Somatovisceral Aspects of Chiropractic: An Evidence-Based Approach and there is nothing in it that would qualify as credible evidence to a science-based thinker. Chiropractic commenters on SBM have told us that modern chiropractic rejects the “subluxation” paradigm and relies on evidence, and I am always willing to look at new evidence and give chiropractors another chance to convince me that a reform movement is really underway, so I looked up the top 10 studies and read them. I was not impressed.
Note: This is a long article with mind-numbing details that will not be of interest to most readers. Feel free to scroll down to the Summary section. You can just read the bold-faced headings describing the claims of each study on the way down.
Also note: For those who want more detail, the “Study #” headings are links to the full text when available online, or to the PubMed citation.
Study #1: Immediate Benefits of Chiropractic Visible on MRI
The full title is “Magnetic Resonance Imaging Zygapophyseal Joint Space Changes (Gapping) in Low Back Pain Patients Following Spinal Manipulation and Side-Posture Positioning: A Randomized Controlled Mechanisms Trial With Blinding.” It was published in the major chiropractic journal, the Journal of Manipulative and Physiological Therapies (JMPT). The authors are from the National University of Health Sciences, a school noted for integrating quackery with medicine. A school that accepts students who don’t even meet their admission requirements and that remedies the lack by teaching them everything they think chiropractors need to know about science in two 8-week sessions of evening classes. Funding was from the NCCAM (in other words, our tax dollars paid for it).
Underlying hypothesis: If the motion of zygapophyseal joints is decreased for any reason (inactivity, repetitive motion, injury), it is hypothesized that they develop adhesions and that both spinal manipulation therapy (SMT) and side-posture positioning (SPP) break the adhesions by “gapping” the joints (separating the articular surfaces, the same process that happens when you crack your knuckles).
Methods: 112 patients with low back pain were randomized to four treatment groups: SMT, SPP, SMT control, SPP control.
First MRI appointment: Patients were placed in a neutral position prior to treatment; the most painful side was always the up side for treatment. An MRI was taken in the neutral position before treatment, then treatment was given followed by a post-treatment MRI.
- SPP Group: SPP, remaining in SPP for 2nd MRI
- SMT control group: Side-posture SMT, followed by neutral positioning for 2nd MRI
- SMT group: side-posture SMT, remaining in side-posture for 2nd MRI
- SPP control: brief SPP followed by neutral positioning for 2nd MRI
Groups 2 and 4 served as control groups because previous studies had shown that returning to the neutral position after treatment resulted in no Z joint gapping difference between the 1st and 2nd MRIs.
Treatment phase: All patients received chiropractic care for 2 weeks, including SMT and other modalities.
Second MRI appointment: After 2 weeks of chiropractic treatment, the second MRI appointment was identical to the first with the exception that the groups were switched (1 with 2 and 3 with 4) so that all subjects were in both an intervention and control group and all subjects were in an SPP and an SMT group during the study.
Results: At the first MRI appointment, the SPP group had greater gapping than the other three groups. (SPP .66 +/- .48 mm, SMT .23 +/-.86 mm, controls .18 +/- .71 mm). At the second MRI appointment, SMT followed by SPP was the only group that showed significantly higher increases in gapping from the first MRI appointment. On pain questionnaires, SMT followed by SPP was the only group that showed significant improvement in pain between the first and second MRI appointments.
Conclusion: They speculated that paraspinal muscles may have relaxed more with SPP at the first MRI appointment, whereas SMT might have resulted in transient increased muscle tightness, and that the reversal of findings at the second MRI appointment may have been because the muscles were more relaxed after 2 weeks of SMT. They conclude that SMT is effective in decreasing pain and increasing gapping and that keeping the patient in the side-posture position for several minutes following SMT may have therapeutic benefit.
My thoughts: The research design was complicated, and trying to figure it out gave me a headache. After reading the report several times, I’m still not entirely sure exactly what they did, why they did it, or what the results mean. We already knew that SMT was effective for low back pain, but we didn’t understand the mechanism. This study attempted to show that the improvement is related to increased gapping of the zygoapophyseal joints, but the authors suggest that muscle relaxation may be the mechanism for both increased gapping and decreased pain, raising the question of whether SMT actually accomplishes anything that other muscle relaxing treatments might not do just as well. Basically, before chiropractic treatment, a single SPP treatment increased gapping more than SMT, and after 2 weeks of SMT treatment, a single SMT treatment increased gapping compared to a single SPP treatment. The suggestion of keeping the patient in a side position following SMT is interesting, but this study was not designed to show whether that would improve clinical outcomes. Since the increased gapping disappears as soon as the neutral position is assumed, its clinical importance is questionable. It would be nice to find out if the hypothesized adhesions are real, but the study didn’t address that.
Study # 2: AMA Recommends Chiropractic Before Resorting to Surgery
This is not a study at all. It is a patient information page from the Journal of the American Medical Association. Under the “Treatment” heading it says:
Many treatments are available for low back pain. Often exercises and physical therapy can help. Some people benefit from chiropractic therapy or acupuncture. Sometimes medications are needed, including analgesics (painkillers) or medications that reduce inflammation. Surgery is not usually needed but may be considered if other therapies have failed.
I don’t think this constitutes a recommendation to try chiropractic before resorting to surgery. It mentions acupuncture in the same breath, and specialty organizations like the American Academy of Orthopaedic Surgeons and the American Academy of Family Physicians do not include either chiropractic or acupuncture in their list of treatment options. As a matter of fact, the WHO guidelines for chiropractic specify that disc herniation with progressive neurological deficits is an absolute contraindication to spinal manipulation treatments. It is really a stretch to claim that the AMA recommends chiropractic.
Study # 3: Chiropractic as Effective as Epidural Injections for Lumbar Disc Herniation
This study by Swiss chiropractors, published in the JMPT, was a comparative effectiveness observational study using data from two databases, one of patients who received nerve root injections (NRIs) and the other of patients who received spinal manipulation therapy (SMT) at a single chiropractic practice. Subjects were 51 matched cases of patients who all had MRI-confirmed lumbar disc herniation. (Note: 28% of asymptomatic people also have these same MRI findings.) Two kinds of SMT were given: intraforaminal disk herniation was treated with modified push adjustment with a kick; paramedian disk herniation was treated with pull adjustment with a kick. They did not explain the rationale for when to push and when to pull.
They found no difference in favorable outcomes between the two treatments.
Other studies have found NRI and SMT effective, but the quality of evidence is poor and many studies fail to differentiate between disc herniation and other causes of low back pain. As mentioned under Study #1, chiropractic manipulation is not widely accepted as a treatment option for lumbar disc disease.
They admit some limitations of their study: it was not a randomized clinical trial, so they themselves point out that
the outcomes of these patients cannot be directly attributed to the specific treatments.
(And yet they go right ahead and attribute.) They did not have sufficient demographic information to assess possible confounders. The sample size was small. Outcome data were collected by telephone interview, a method that tends to produce more positive responses than questionnaires. Outcomes were subjective patient reports, not objective measurements. The groups were not comparable: the NRI group had higher pretreatment pain levels. There was an obvious source of bias: patients or referring physicians had chosen which treatment they wanted to try. Also, the NRI was done at a single appointment, while chiropractic patients were seen for multiple treatments. The time spent with the provider is a probable confounder.
They said SMT was slightly less expensive than NRI, but they admitted that they were only looking at direct procedure costs and could not determine overall cost-effectiveness.
This study was not even randomized. It would have made a lot more sense to have done a prospective randomized study and to have ruled out as many confounders as possible. Because of the flaws in this study, no conclusions can be drawn from it.
Study # 4: Chiropractic Lowers Blood Pressure
This was a study from Sherman College of Chiropractic published in the Journal of Chiropractic Medicine entitled “Blood pressure changes in African American patients receiving chiropractic care in a teaching clinic: a preliminary study.” There were only 24 subjects, all black patients diagnosed with pre-hypertension (120-139/80-89) or hypertension stage 1 (140-159/90-99), with or without medication. There was no control group, and the subjects were not a random or characteristic group of hypertensive patients, but a sample of convenience. Chiropractic did NOT lower blood pressure. The results were negative! BP did not change significantly after chiropractic care. They weren’t happy with that result, so they tried throwing out 4 data points from patients with “outlier” high BMIs, and lo and behold! The remaining data yielded a statistically significant reduction in BP (diastolic pressure only). I don’t think throwing out the outliers was justified; I think it constituted illegitimate data mining. And even for the tweaked dataset, the difference was not clinically significant. The diastolic pressure only dropped from 81.5 to 77.1 and the systolic pressure was unchanged.
Study # 5: Chiropractic Best Option for SI Joint Pain
“Treatment of the sacroiliac joint in patients with leg pain: a randomized-controlled trial” was a single-blinded randomized trial from the Netherlands, published in the European Spine Journal. It was a small trial: a total of 51 patients with leg pain attributed to the SI joint were randomized to one of three treatment groups: physiotherapy (exercise), manual therapy (manipulation with high-velocity thrust technique), and intra-articular steroid injections. They set simple criteria for success and failure: success was complete relief or improved VAS (visual analog scale) pain score; failure was dropping out because of worsening complaints or a VAS score that did not improve. The PT group had 12 failures and 3 successes, the manual therapy group had 5 failures and 13 successes, and the injection group had 9 failures and 9 successes. The magnitude of the improvement was small. On the 10-point VAS of pain, PT improved the score by an average of 0.4 points, manual therapy by 1.9 points, and injection by 0.7 points. In their opinion, a clinically meaningful difference was 1.3-2 points. I’m not sure I agree.
They concluded that manual therapy was the treatment of choice for this select group of patients with SI-related leg pain, but cautioned that the findings may not apply to patients not meeting the specific criteria for inclusion in this study and that because of the small sample size, their results need to be confirmed by a larger study.
The patients were referred by general practitioners with the question whether they had a radiculopathy due to a herniated lumbar disc. It can be difficult to differentiate disc pain from SI pain, and some of the tests have questionable inter-observer reliability.
They mention that they had difficulty recruiting patients willing to be randomized. 23% had an overt favorite. 10 wanted an injection, 5 did not want an injection, 1 wanted manual therapy, and 3 wanted PT.
There is no mention of chiropractors in the study. They did not specify who provided the manual therapy.
The findings of this study contradicted other studies. A 2012 study showed that lumbar manipulation was more effective than SI manipulation. Other studies have shown a greater response to intra-articular injections (67%) than the 50% that this study found. And previous studies have found PT to be effective.
Study # 6: Neck Adjustments Immediately Improve Joint Position Sense
A study from Korea published in the Journal of Physical Therapy Science evaluated 30 normal adults who were divided into two groups (no mention of randomization): the test group was given cervical joint manipulation and massage and the control group got massage only. Reduction in cervical range of motion (ROM) is thought to reduce joint position sense and possibly to be related to dizziness; this study investigated whether increasing ROM in normal volunteers by neck manipulation would improve joint position sense (JPS). Researchers showed patients how to adopt six exact neck positions and then were tested with an inclinometer to see if they could adopt the same positions without instruction. The test group improved significantly on all six tests; the control group didn’t. In a comparison of the test group to the control group, two tests showed a significant difference; the other four didn’t. Errors declined from 2.5 to 3 degrees pre-treatment to 1-1.2 degrees post-treatment. The authors recommend that:
when treating patients with reduced JPS due to decreased ROM, persistent neck pain, or whiplash injuries, the combined application of cervical joint manipulation and massage [be] used.
That conclusion is not supported by the data. A study of JPS on normal volunteers with a normal ROM can’t be used to make clinical recommendations for patients with neck pain and decreased ROM. They didn’t even measure ROM; they just measured people’s ability to assume a previously taught position. The clinical significance of improved JPS is unknown. If they think it is the mechanism by which manipulation works, they need to study JPS in patients with neck pain.
Of concern: neck manipulation carries a small but serious risk of stroke. Question: would gentle mobilization and massage get the same results with less risk?
Study # 7: Chiropractic Better than Medical Care Alone for Back Pain
A randomized controlled trial by American chiropractors entitled “Adding Chiropractic Manipulative Therapy to Standard Medical Care for Patients With Acute Low Back Pain: Results of a Pragmatic Randomized Comparative Effectiveness Study.” Subjects were 91 military men and women between 18 and 35 years of age with acute low back pain (LBP). All patients got standard medical care and 45 were randomly assigned to also get chiropractic treatment. Standard medical care (SMC) was a single visit that might include any or all of the following: a focused history and physical exam, diagnostic imaging as indicated, education about self-management including maintaining activity levels as tolerated, pharmacological management with the use of analgesics and anti-inflammatory agents, and physical therapy and modalities such as heat/ice and referral to a pain clinic. Chiropractic treatment (CMT) consisted of 5 visits for high velocity/low amplitude (HVLA) manipulation plus whatever else the chiropractor chose to do (education, exercise, mobilization, etc).
Results: 73% of the SMC + CMT group reported improvement compared to 17% in the SMC-only group. Satisfaction on a 10-point scale was 8.9 and 5.4 respectively. There were two adverse events after chiropractic treatment.
Problems:
- Followup rates were 85% for the standard care group and 63% for the combined care group.
- Participants had higher expectation of helpfulness for the combined medical/chiropractic treatment than for standard care alone. This is a clear source of bias.
- Pain, functional status, and patient satisfaction were measured by subjective answers to questionnaire; no objective measurements were done. They did not even track use of pain medication during the trial or speed of return to full duty.
- Care was not standardized. Improvement could have been due to some other component of treatment.
- Multiple visits to chiropractor were compared with a single visit for standard care.
- This was a select group of healthy young patients; results might not apply to the general patient population.
The design of this trial practically guaranteed that it would favor chiropractic. Pragmatic comparative effectiveness trials of CAM treatments are inherently biased to favor CAM. They allow the nonspecific effects of provider/patient interactions to shine. They are not designed to test whether a CAM treatment is superior to placebo; they are a way of bypassing that essential step.
Study # 8: Spinal Adjustments Relieve Muscle Pain Instantly
From the JMPT. 33 healthy young adults with myofascial trigger points in the gluteus medius and infraspinatus muscles were randomized and treated with SMT targeted to the C5-6 spinal segment or with sham SMT. The pressure pain threshold increased in the treatment group. Chiropractic theory says that SMT relieves pain by improving nerve function. Does increased pressure pain threshold mean the nerves in the infraspinatus muscle are functioning better? We don’t know; the study didn’t address that. Does the finding have any clinical importance for patients with muscle pain? The study didn’t address that either. (Our assistant editor Paul Ingraham has written much about the idea of myofascial trigger points, skeptically.)
Study #9: Cervical Disc Herniation Eased by Chiropractic
A study by Swiss chiropractors published in the JMPT entitled “Outcomes from Magnetic Resonance Imaging-Confirmed Symptomatic Cervical Disk Herniation Patients Treated with High-Velocity, Low-Amplitude Spinal Manipulative Therapy: A Prospective Cohort Study with 3-Month Follow-Up.” This was an uncontrolled study of 50 patients. 55% were “improved” at 2 weeks, 69% at 1 month and 86% at 3 months, according to the patients’ own global impressions; there were no objective measurements. Improvement was greater in patients with acute pain than in those with chronic pain. The treatment is exactly the kind of neck manipulation that has been associated with strokes.
How many of these patients would have improved with no treatment at all, due to the natural course of the disease? The authors say it is virtually impossible to extract reliable figures on the natural history of this condition, so a control group is crucial to interpreting the significance of their findings, and it’s hard to understand why they failed to use one. While double blinding with a credible placebo control would be difficult, they might at least have compared HVLA manipulation to gentle mobilization with exercise and physical therapy modalities like heat.
Study # 10: Chiropractic Thaws Frozen Shoulder Syndrome
From the Journal of Chiropractic Medicine. This was a case series of 50 consecutive patients diagnosed with frozen shoulder syndrome (FSS) and treated with a series of chiropractic manipulation treatments of the cervical and thoracic spine, plus a novel chiropractic method, the OTZ Tension Adjustment, which involves a “skull glide” and an HVLA thrust to correct occipito-atlantal subluxation. There was no control group. The findings:
Many of these patients’ complaints seemed to improve or resolve within 1 month of presentation, whereas, in general, it is thought that FSS symptoms can persist for 2 years or more.
This is junk science.
- There is no credible mechanism by which anything in the spine could cause FSS.
- There is no such thing as a chiropractic subluxation.
- Treating problems elsewhere in the body by adjusting the 1st cervical vertebra doesn’t work and is rejected even by most chiropractors.
- “Skull glide” is not defined and I couldn’t find anything about it on the Internet. Probably a bogus untested maneuver.
- The OTZ adjustment has never been tested and there is no reason to think it should work for anything, especially in the shoulder. The founder of the system claims it works for frozen shoulder, migraine, vertigo, insomnia, Bell’s palsy, ADHD, posture, and other conditions.
- The lead author is the owner and founder of the OTZ system.
- All patients had been treated by the lead author.
- No control group.
- Frozen shoulder usually resolves in a matter of months with physical therapy; even without treatment, it resolves within 24 months.
- Manual manipulative treatment of the shoulder itself has been tested and found helpful for FSS, but manipulation of the spine has not.
It would have been so simple to use a control group, even if it were only to compare chiropractic treatment with and without OTZ. The lead author invented OTZ and proceeded to treat thousands of patients without bothering to test whether it worked.
This amounts to Tooth Fairy Science squared: it is a study about treating an imaginary subluxation with an imaginary treatment.
Summary
To recap, my evaluation of these “10 best studies” showed:
Study 1. Bewilderingly complicated design, clinical relevance of findings not established.
Study 2. Not a study
Study 3. Small study with numerous limitations and evidence of bias; the authors themselves said outcomes could not be attributed to treatment.
Study 4. Negative study misrepresented as positive
Study 5. Small study with evidence of bias; results incompatible with previous studies
Study 6. Small study of joint position sense in normal people, with questionable clinical relevance. Data do not support conclusions.
Study 7. Pragmatic comparative effectiveness trial designed to guarantee it would favor chiropractic
Study 8. Small study with questionable clinical relevance
Study 9. Uncontrolled study
Study 10. Uncontrolled case series; junk science intended to promote the OTZ System, whose owner and founder is the lead author and treating chiropractor.
Questions remain
Is chiropractic effective? Numerous studies have established that SMT is:
- Effective for certain kinds of low back pain but not more effective than other treatments.
- Effective for certain types of neck pain but only when accompanied by exercise, and not more effective than mobilization.
- Not effective for any other condition.
None of these 10 studies would change those conclusions.
How does chiropractic work? 1,6,and 8 addressed possible mechanisms for effects of SMT. Chiropractic was founded in 1895 and we still don’t know how it works. None of these studies provides a credible answer.
Why aren’t they doing better studies?
How do these studies measure up on Bausell’s 4-point checklist? Not very well:
- Subjects randomly assigned to a CAM therapy or a credible placebo control? At best, 1 out of 10 studies.
- At least 50 subjects per group? 0 out of 10 studies.
- Less than 25% dropout rate? 9 out of 10 studies.
- Publication in a high-quality, prestigious, peer-reviewed journal? 0 out of 10 studies.
Standards for conducting well-designed clinical trials are common knowledge today. Why is anyone still doing uncontrolled studies or using inadequate controls? Why is so little research on chiropractic of sufficient quality to merit publication in high quality journals?
Conclusion
If these 10 studies are the best chiropractic research from 2013, that means the quality of chiropractic research is appallingly poor.
This article was originally published in the Science- Based Medicine Blog.
