A very good guideline is one in which many of the dimensions are addressed, and some of the recommendations are linked to evidence levels. Objectives and rationale for development are often clearly defined but may be lacking in other areas, such as application (eg, outcome measures, targets, risks, and benefits). These are generally well produced and useful for practicing clinicians and are recommended.
In a fair guideline, some of the dimensions are addressed, but there are some major omissions, often in terms of levels of evidence, literature search strategy, clarity, risks, and benefits. Often these documents are local adaptations of other guidelines. Information can sometimes be used as a general reference if user-friendly materials are incorporated but are generally not very useful as guidelines. These guidelines are recommended under special circumstances.
A poor guideline is one which most of the dimensions are not well addressed, if at all. Often, it is unclear who produced these documents, and there is no description of the individuals involved. Levels of evidence and literature search strategy are rarely included, and there is no description of the methods used to formulate the recommendations. These guidelines are of little use to practicing clinicians and are not recommended.
Recognizing that recommending guidelines based on the quality of the process by which they were produced and the evidence used in their development would be controversial, we felt it was extremely important to develop a rigorous and objective scoring methodology. Fellows from the Department of Family and Community Medicine at the University of Toronto and community-based family physician volunteers from the OMA were brought together in 5 workshops. Each workshop included approximately 20 participants and consisted of a half-day session on the objectives of the GAC, a detailed review of the Appraisal Instrument, and a hands-on session where all participants evaluated the same guideline. Scores were then openly declared, and a discussion held on discrepancies identified in the assessments in an attempt to standardize the process. At the end of the session, interested participants were provided with an additional 5 guidelines to assess in the subsequent 2 weeks. The resulting appraisals were evaluated for consistency and inter-rater reliability (results indicate that using the instrument as an initial filter to determine the best-quality guidelines in each clinical area is a valid approach). To date, 45 assessors have been trained and are reviewing guidelines on an ongoing basis. Each guideline is evaluated a total of 3 times by independent assessors. Those guidelines that have been selected for recommendation in a particular clinical area are then reviewed for clinical relevance and applicability to the Ontario context. More than 250 published guidelines have been identified and distributed to physician assessors in the clinical areas shown in Table 2.
Reformatting
The GAC is in the process of determining the user-friendliness of recommended guidelines. Not infrequently, guidelines that are found to be the most evidence-based and objective are hundreds of pages in length and would be extremely burdensome for the average family physician to use. We anticipate that guidelines found to be of excellent quality but not convenient for use in clinical practice will need to be reformatted into user-friendly summaries. Volunteer physicians from the community will be asked to evaluate such summaries and provide feedback for improvement.
Dissemination
Once the best-quality guideline(s) on a topic are identified and reformatted as necessary; we intend to mount them on the GAC Web site (www.gacguidelines.ca) for use by the profession and the general public. Table 4 shows the results of the guideline selection process for the first 10 clinical areas. The process for choosing guidelines is transparent so that practicing physicians can determine for themselves the usefulness and applicability of the recommendations. Only the most rigorously developed guidelines will be posted on the Web site in the form of structured summaries, although interested clinicians can obtain the outcome of nonrecommended guideline appraisals on request.
Continuing medical education literature on dissemination strategies indicates that a single method, such as posting information on a Web site or mailing guidelines to clinicians has a minimal effect on changing medical practice.6 The GAC is currently considering a number of options to enhance the dissemination of the best available guidelines. Since Ontario health data on diagnostic testing, hospitalization records, and office visits are collected provincially, it could be possible to measure clinical outcomes following the dissemination of evidence-based guidelines. We are currently working with provincial groups to disseminate guidelines through medical school continuing medical education (CME) division programs, peer presenter programs, small group CME programs, outreach facilitation programs, and a peer assessment program run by the provincial licensing body.