Development of the Standardized Patient Scenarios
Primary care and psychiatric faculty from each study center collaborated to develop the initial case scripts. These presentations were chosen to address 2 ends of the spectrum of common presentations for depression in primary care: obvious mental health distress versus more subtle distress of a sufficient degree to be associated with dysfunction as demonstrated in other studies.24 The scenarios were evaluated for feasibility, credibility, and internal consistency and were refined during guided focus groups of community physicians held in each study center.25 No pilot test or focus group physicians were study participants.
The actors portrayed the roles of either a 45-year-old corporate loan officer or a 26-year-old data entry clerk (Table 1). In both cases, a recent move required initiation of contact with a new care provider. According to the scenarios, the patient’s insurance was in transition because of a new job, but this was not scripted to be a barrier to care in either case. Encounters were paid for in cash, with the standardized patients indicating they would submit their own insurance claims. Standardized patients assigned to scenario A volunteered sleep and concentration difficulties as their chief complaints. No other DSM criteria were volunteered. If depression was not mentioned as a possible diagnosis to the patient enacting scenario B by the end of the second visit, the prompt, “I’ve had a really tough year,” was delivered by the patient. We did this to increase recognition of depression, allowing us to obtain information on the management approaches taken by all participating physicians. At the second visit, patients enacting both scenarios reported their symptoms were 50% better regardless of the treatment suggested during the first visit.
None of the actors had a personal history of depression. The standardized patients refused an extensive medical work-up at the time of the visits, stating they would prefer to wait until their insurance status was resolved. The exception was urinalysis, if it was part of the physician’s routine orders.
Measures
An evaluation checklist with dichotomous (yes/no) responses was developed to allow standardized patients to assess how participating physicians pursued the presenting complaints and the 9 criteria for major depressive disorder listed in the Diagnostic and Statistical Manual of Mental Disorders, Third Edition, Revised (DSM-III-R). The checklist was pilot-tested in each study center to ensure that each item was readily observable and accurately scored by the standardized patients. Audiotape recorders with high-fidelity microphones, concealed in briefcases or book bags, were used to record the physician-patient interactions. After each encounter, the standardized patient completed the checklist, reviewing the audiotape when necessary.
At each center, trained nonphysician project staff reviewed 20% of all audiotaped encounters for accuracy and reproducibility. Mechanical failure occurred in 3% of encounters. Agreement between the standardized patient and the project staff ranged from 79% to 100% for recorded encounters. The low end of this range concerned queries about mood, where the standardized patients were more likely than trained staff to score a question such as “How have you generally been feeling?” as an assessment of mood. Thus, standardized patients were at risk for overestimating the proportion of physicians asking about mood. Checklist data scored by the standardized patients were used in these analyses, since they represented the most complete and consistent record of the encounters.
Standardized Patient Training
Twenty-five actors were recruited from local medical education programs at the 3 participating institutions. All standardized patients were within 120% of their ideal body weights, and their clothes were consistent with the cases being portrayed. Scenario development and standardized patient training were initially done at the northern New England site, and training videotapes and materials were disseminated to the other 2 study centers. During the first phase of training, 24-hour didactic training sessions were held for each case, to inform the actors about the detailed scenario and the skills necessary to enact it. At the end of each session, actors observed videotaped interactions of themselves and completed study checklists, which were then compared and discussed until consensus was achieved on scoring. During the second phase of training, each actor visited 3 nonparticipating clinicians. None of these pilot physicians had any specific knowledge of the study and were instructed to respond to the patient as they would to any patient making the same request. Immediately after each videotaped session, the actors completed an evaluation checklist. The videotapes were then scored by the trained project staff at each study center using the same checklist, and any discrepancies in scoring or replicating the scenario were discussed. Twenty-four of the 25 actors consistently replicated the scenario and accurately assessed the providers (greater than 95% agreement with the trained observer) and were allowed to begin study visits.