The American Academy of Family Physicians journal American Family Physician (AFP) has a feature called Journal Club that I’ve mentioned before. Three physicians examine a published article, critique it, discuss whether to believe it or not, and put it into perspective. In the September 15 issue the journal club analyzed an article that critiqued the process for developing clinical practice guidelines. It discussed how two reputable organizations, the United States Preventive Services Task Force (USPSTF) and the American Academy of Pediatrics (AAP) looked at the same evidence on lipid screening in children and came to completely different conclusions and recommendations.
The AAP recommends testing children ages 2-10 for hyperlipidemia if they have risk factors for cardiovascular disease or a positive family history. The USPSTF determined that there was insufficient evidence to recommend routine screening. How can a doctor decide which recommendation to follow?
What Are Clinical Practice Guidelines?
They are “cookbooks” developed for the convenience of doctors in practice. A group of experts asks a practical question (for instance, should we screen the general population for prostate cancer with a PSA test?) and does the dirty work, plowing through mountains of evidence so when a doctor in practice asks that question, a handy science-based answer will be available to him. There is a National Guideline Clearing House that has compiled 2476 of these guidelines, with as many as 20 guidelines on a single topic. They don’t provide any oversight of quality or of how the guidelines were developed. In general, guidelines from specialty groups tend to be methodologically less sound and to recommend a more aggressive strategy than guidelines from primary care organizations. I recently wrote aboutprostate cancer screening, where most groups now recommend against routine PSA testing but urologists strongly disagree. Another problem is that published guidelines are not always current; they must be updated as new evidence comes in.
Comparison of the USPSTF and AAP Guideline Development Process
Why did the USPSTF and the AAP disagree? This table offers some clues:
| Standard | USPSTF | AAP |
| Clear description of methods used to identify and analyze scientific data | Yes | No |
| Method of updated literature review that is referenced and outlined | Yes | No |
| Standard process for appraisal and grading the quality or sufficiency of the evidence | Yes | No |
| Conflicts of interest rigorously vetted | Yes | No |
| External peer and public review | Yes | No |
The Journal Club participants felt the USPSTF process was “best in class,” and they chose to follow the more rigorous, more conservative, USPSTF recommendation over the AAP recommendation for childhood lipid screening.
Problems with the Process
All too often, there is no good clear-cut evidence to answer the question, so guidelines fall back on consensus recommendations from a group of experts. A review of American Heart Association/American College of Cardiology guidelines found that only 11% were based on multiple randomized trials or meta-analyses, and nearly half were based only on experts expressing their opinions. Also, 56% of guidelines had an author who disclosed a conflict of interest. Worse yet, 87% of guidelines had ties to the pharmaceutical industry.
In one documented case, a prominent nonprofit organization received $11 million dollars from a pharmaceutical company and then placed 6 members with ties to that company on a 9-person committee charged with determining the role of the company’s drug in a guideline.
Another problem is lack of transparency. Published guidelines seem to imply unanimity, but individual committee members may have disagreed. It would be helpful to know whether the vote was 9-0 or 5-4. And why.
Improving the Process
The Institute of Medicine (IOM) has proposed these 8 standards:
- Complete transparency
- Conflict-of-interest disclosure
- Multidisciplinary member composition
- Systematic literature review meeting IOM standards
- Clear and consistent rating and description of evidence
- Recommendations articulated in detail and in a standard form
- External review by the full spectrum of stakeholders
- Appropriate updating of guidelines
There is no requirement to follow these standards, and no way to enforce them. The American Cancer Society has said it will follow them. The WHO has also proposed its own rigorous 19-step process.
Conclusion
The Journal Club analysis concludes with a summary of key points, including:
- There are no controlled trials demonstrating that lipid screening in children improves long-term health outcomes.
- The level of evidence of clinical practice guidelines should be reviewed before widespread implementation.
I would add: Clinical practice guidelines are very useful and the USPSTF guidelines are the most trustworthy.
This article was originally published in the Science-Based Medicine blog.
