Eighty percent of respondents strongly agreed that COI disclosure is important when interpreting study results, 62% reported always reading the disclosure slide during academy or other meeting presentations, and 41% reported always using this information when deciding how to interpret scientific data.
Seventy-five percent of respondents thought the study—an academic-center case series with significant results in favor of the pharmaceutical company funding the study—was biased (42% indicated biased with merit, 33% biased without merit). Twenty-three percent thought the study was possibly biased, but likely trustworthy given the academic institutional affiliation. When the study setting was changed to community hospital, 95% thought the study was biased (51% biased with merit, 44% biased without merit). With the same study performed at an academic center, and no statistically significant results (not in favor of the pharmaceutical company funding the study), 88% thought the study had merit (46% biased with merit, 42% unbiased with merit).
When the study design was changed to a randomized controlled trial (level I evidence) conducted at an academic center with negative results, an overwhelming 95% of respondents thought the study had merit (33% biased with merit, 62% unbiased with merit). Given the same study design at an academic center, with positive results, 78% still thought the study had merit (39% biased with merit, 39% unbiased with merit). An additional 18% thought the study was biased, but still likely trustworthy given the academic institutional affiliation. Finally, given a randomized controlled trial and positive results, but with the research setting a small community practice, 90% thought the study had merit (51% biased with merit, 39% unbiased with merit). The percentage of respondents who found the study biased and likely without merit increased from 3.7% to 9.5% when the institutional affiliation changed from academic to community.
Discussion
As governmental funding sources become increasingly limited, the role of industry sponsorship of orthopedic research has grown. Potential drawbacks and biases of such research support have been well described—most notably, increased positive result reporting, suppression of results that may be disadvantageous to the industry sponsor, and biased study designs.2-8 However, the extent to which financial COIs affect the orthopedic medical community’s interpretation of the literature has never been quantified. To our knowledge, the present study is the first to quantify the impact of reported COI on study interpretation.
Our goal was to examine how reported financial COIs influence the interpretation of the literature by the orthopedic medical community. Moreover, we wanted to determine the perceived reliability of the data when variables (study design, institutional affiliation, positive vs negative results) were changed. The results of our survey indicate that, when a financial COI is reported, study reliability is perceived as highest when negative results were found.
Our survey noted a discrepancy between the documented importance of the hypothetical research team’s COI disclosure and the use of such disclosures when interpreting study results. Eighty percent of respondents agreed that COI disclosure is important when interpreting study results, but only 62% reported always reading disclosures, and even fewer (41%) reported always using the information when interpreting results. It is unclear exactly why this trend exists, as one would expect the percentages to be more similar. These particular survey questions were formed around using COI disclosures when interpreting study results during academic presentations at national meetings and not during the review of published literature. It is possible that positioning the COI disclosure at the beginning of a presentation has an effect, but only 3.7% of respondents indicated they seldom remembered the disclosure by the end of the presentation. The results of our survey may have varied if the questions had targeted reading and interpreting the literature.
Interestingly, the results of these survey questions tended to be more consistent with rates of reported financial COI by presenters at national orthopedic meetings. A study published in the New England Journal of Medicine found that less than 80% of orthopedic surgeons reported their disclosures at a large annual meeting (AAOS), even when the disclosure involved payments pertinent to the research they were presenting.5 When the payments were indirectly related to the research, the percentage of surgeons reporting disclosures was 50%, almost the same as the disclosure rate for unrelated payments.5
When the study was changed to a level I randomized controlled trial, more survey respondents found it to be less biased and have more merit. Although it would seem intuitive for a study with a higher level of evidence to carry more clinical value during interpretation, this may not hold true in the setting of industry-sponsored clinical trials. Several studies have documented a significant association between the reporting of positive results and industry-sponsored randomized clinical trials. In 2008, Khan and colleagues3 examined 100 orthopedic randomized clinical trials reported in 5 major orthopedic subspecialty journals over a 2-year period. The association between industry funding and favorable outcome in all original randomized clinical trials was strong and significant (P < .001). This is not surprising, given the amount of time and money required for a well-designed clinical study. Commercial products with preclinical promise are pushed to testing in a clinical trial, whereas resources would not be wasted on products lacking preclinical merit.