Major Finding: Only 14% of 4,218 individual recommendations in 41 Infectious Diseases Society of America clinical practice guidelines are based on level I evidence such as that from randomized clinical trials, while more than half are based on level III evidence, such as that from expert opinion or descriptive studies.
Data Source: A review of 41 current IDSA clinical practice guidelines aimed at assessing the quality of evidence on which each recommendation is based.
Disclosures: Dr. Lee and Dr. Vielemeyer reported that they had no relevant financial disclosures.
More than half of the current recommendations in practice guidelines concerning infectious disease are based on evidence derived only from expert opinion or descriptive studies, according to Dr. Dong Heun Lee and Dr. Ole Vielemeyer of Drexel University, Philadelphia.
Only 14% of the 4,218 individual recommendations included in 41 Infectious Diseases Society of America (IDSA) guidelines published in 1994-2010 are based on the highest-quality, or level I, evidence, such as that from randomized controlled trials, Dr. Lee and Dr. Vielemeyer reported.
“Guidelines can only summarize the best available evidence, which often may be weak. Thus, even more than 50 years since the inception of evidence-based medicine, following guidelines cannot always be equated with practicing medicine that is founded on robust data,” they noted.
“Physicians and policy makers should remain cautious when using current guidelines as the sole source guiding decisions in patient care.”
The study authors assessed the quality of evidence underlying 41 of the 52 IDSA guidelines currently available, which cover a wide range of topics and use an IDSA evidence-grading system. About half of these 41 guidelines are new and half are updates of earlier guidelines.
In addition to the highest-quality (level I) evidence, the IDSA grading system designates evidence from well-designed, but nonrandomized clinical trials, from cohort studies, from case-controlled analytical studies, or “dramatic results from uncontrolled experiments” as intermediate-quality (level II) evidence. The lowest-quality (level III) evidence is that “from the opinions of respected authorities based on clinical experience, descriptive studies, or reports of expert committees,” the investigators said.
Dr. Lee and Dr. Vielemeyer identified 4,218 individual recommendations among the 41 guidelines that could be charted according to the strength of the recommendations and the quality of the evidence supporting them. Only 14% were supported by level I evidence, 31% by level II evidence, and 55% by level III evidence (Arch. Intern. Med. 2011;171:18-22).
For example, greater than 80% of the recommendations concerning blastomycosis, which were published in 2008, were based on level III evidence and did not have any level I support. The findings were the same for recommendations concerning sporotrichosis, which were published in 2007.
The investigators also assessed the extent to which the quality of evidence has improved over time by selecting five guidelines that had recently been updated and comparing them with their respective earlier versions. The updates did include evidence from more studies, as well as evidence from more recent studies, than did the earlier guidelines. “However, only two updated guidelines had a significant increase in the number of level I quality-of-evidence recommendations; most additional recommendations were supported by level II or III quality of evidence only,” Dr. Lee and Dr. Vielemeyer said.
In addition, “we came across imprecisions on more than one occasion and for more than one guideline, including illogical, erroneous, or missing references for recommendations and their associated grades,” they added.
These findings are particularly concerning because guidelines are used not only for decision making in clinical practice but also “as benchmarks in the appraisal of quality of care provision,” they said.
“We believe that the current clinical practice guidelines released by the IDSA constitute a great and reliable source of information that should be used. However, in circumstances when patient outcome is less than desirable, or when colleagues use diagnostic or therapeutic choices not included in the recommendations, it is prudent to remember that many of the individual recommendations are not supported by solid evidence.
“In such cases, we encourage reviewing the primary literature and using one's clinical judgment rather than relying solely on recommendations,” Dr. Lee and Dr. Vielemeyer concluded.