Methods
As part of a larger clinical trial to improve preventive practice, we conducted a cross-sectional study to provide a point estimate of preventive performance in capitation primary care practices. We chose the preventive maneuvers from the Canadian Task Force on the Periodic Health Examination.11 According to their classification system, there is randomized clinical trial evidence to support “A” level (highly recommended) maneuvers and cohort and case controlled studies to support “B” (recommended) maneuvers. The task force also reviewed the quality of evidence for maneuvers that should not be done and identified these as “D” level maneuvers. Eight A and B level recommendations and 5 D level recommendations were identified by a panel of practicing family physicians. Selection criteria included the need to represent a broad spectrum of preventive interventions for both men and women patients of all ages, and the need to address diseases that were clinically important. The 8 recommended and 5 inappropriate maneuvers chosen for our study are listed in Table 1.
This study was conducted in 72 community-based health service organizations (HSOs) in Ontario located at 100 different sites primarily in the Toronto, Hamilton, and London areas in the spring of 1997. The Ottawa Civic Hospital research ethics committee approved our study.
Data Collection
Practice and physician characteristics were collected using a self-administered questionnaire to which 96% of 108 participating physicians responded (Table 2 has the questionnaire items). Preventive performance at the physician and overall practice level was determined using a chart audit.
Chart Audit. Patient charts were eligible for inclusion in the medical audit if they were for patients who were aged 17 years or older on the date of last visit and had visited the HSO at least once in the 2 years before the audit. The variables collected from the charts included demographic and patient characteristics as well as indicators of performance of preventive maneuvers.
The chart auditors obtained a list of patients within an HSO practice group of physicians and then randomly selected charts using computer-generated random numbers. The patient list was either constructed by the auditors or by using the medical office record computer system. The list included all rostered and nonrostered patients. Unique chart numbers or numeric identifiers were assigned to each patient. The required number of charts was randomly selected from the sampling frame, the chart was pulled, and eligibility for inclusion was determined. The auditors proceeded to find charts at random from the sampling frame until they obtained 100 eligible charts per practice.
To verify the quality of the data entered from the 100 randomly selected charts and to measure the inter-rater reliability between auditors, 20% of each HSO’s audited charts were independently verified by another auditor. If coding discrepancies were found in more than 5 of 20 charts, the entire 100 charts were audited and verified again.
Data Analysis. Our analysis with SPSS software version 8.0 (SPSS Inc, Chicago, Ill) focused primarily on calculating the extent to which each preventive maneuver was being performed according to the recommendations of the Canadian Task Force on the Periodic Health Exam. An index of “up-to-datedness” was computed for each physician and practice as the number of A and B preventive measures done divided by the number of eligible A and B measures. In addition, an index called “inappropriateness” was computed in the same manner to represent the D measures.
Frequencies and descriptive statistics were generated on all variables, and each variable was checked for data entry errors and inappropriate or illogical responses. Means and standard deviations were computed for continuous variables and frequency distributions were computed for categorical variables, such as sex and age group. In addition, chi-square tests were used to compare the background characteristics of participating and nonparticipating HSO physicians. Ninety-five percent confidence intervals were calculated for the mean preventive indexes. Finally, kwas computed as a measure of reliability between the 2 chart auditors.
The ICC was calculated for sample cluster means12 of up-to-datedness and inappropriateness. The practice characteristic data revealed that the mean cluster size in terms of number of physicians per practice was 2.8 with a variance of 3.6 and a total of 106 physicians across 46 practices. To determine the between-subjects (practices) variance (sb2) and within-subjects (practices) variance (sw 2) for the ICC calculation, the one-way analysis of variance (ANOVA) procedure was run on both measures (up-to-datedness and inappropriateness) as well as each of the preventive maneuvers separately.13 ICC (D) was computed from the F statistics of the one-way ANOVA and the adjusted cluster size as follows: D = F-1/F+n0-1 where n0 is the mean practice size ([2.8] - [practice variance (3.6)/106 physicians]).