Premarketing approval of cardiovascular devices by the Food and Drug Administration often rests on a shaky foundation, according to a review.
Most of the clinical studies the FDA has relied on to approve CV devices are neither blinded nor randomized. About half are not controlled or use only historical controls, which produces biased results favoring the devices, the study investigators reported. In addition, most of the studies use surrogate instead of clinically meaningful end points, use composite instead of individual outcome measures, exclude data on patients who have unfavorable outcomes, and are performed in subjects not representative of the patient populations that will be using the devices.
Moreover, the majority of such FDA approvals have rested on the results of a single study, reported Dr. Sanket S. Dhruva and associates at the University of California, San Francisco
The public assumes that the FDA has a “rigorous device approval process, and strict standards for cardiovascular devices.” Yet the type and quality of the evidence on which the FDA bases its approval have never been systematically examined until now, the investigators noted.
They reviewed the 123 clinical studies underlying FDA approval of 78 cardiovascular devices between 2000 and 2007.
The mean number of studies supporting each approval was only 1.6; fully 65%of the device approvals were supported by only a single study. Most approvals did not cite even one blinded or randomized study. Overall, only 27% of the supporting studies were randomized and only 14% were blinded.
Nearly half of the studies supporting FDA approval failed to include a control group for comparison. Of those that did include a control group, retrospectively selected controls were commonly used, which biases the results in favor of the device, the authors wrote.
Of the supporting studies, 14% did not even state a primary end point. Moreover, “the vast majority of end points were surrogates, which may not be reliable predictors of actual patient benefits,” Dr. Dhruva and colleagues said.
Most studies assessed composite rather than individual outcomes, which in cardiovascular trials “have been shown to compromise individual end points that often vary in clinical significance and do not contribute equally to the composite measure,” the researchers added.
Many studies excluded data from lead-in periods, which effectively excludes subjects who have immediate unfavorable outcomes. Most also showed large discrepancies between the number of subjects enrolled and the number included in final analyses.
In all, data on 10,352 subjects were excluded, which constitutes nearly a third of the total study population; 20% of the studies did not even report the number of participants.
In more than one-third of the device approvals, “we were not able to ascertain that even one study had been conducted in the United States. This results in uncertain generalizability of approved medical devices to the U.S. population,” Dr. Dhruva and associates said (JAMA 2009;302:2679-85).
In addition, many devices were approved “using a post hoc analysis of data,” which can bias the results in favor of the device, they said. In one notable example, a cardiovascular device was approved by the FDA based “wholly on a post hoc analysis for a single subgroup” assessed in a single study.
“The importance of the 'seal of FDA approval' cannot be overstated. Many manufacturers immediately encourage widespread use of their devices based on FDA approval through direct-to-consumer advertising, detailing to physicians, and continuing medical education venues,” the investigators noted.
Such devices also are commonly used for nonapproved indications. “For example, Medicare data show that 69% of current drug-eluting stent use is 'off-label,'” Dr. Dhruva and colleagues noted.
The study findings are particularly disturbing given that FDA device approval effectively preempts consumers from bringing lawsuits related to problems with device safety or effectiveness. Moreover, manufacturers are not required to seek out and report device malfunctions, “so device-related adverse events are substantially underreported,” the researchers said.
“The bar for evidence of benefit should be higher for devices [than for drugs] because they are implanted and cannot simply be discontinued, as drugs can be,” they added.
Disclosures: Dr. Dhruva's associate in this study, Dr. Rita Redberg, reported being a member of the FDA Circulatory System Devices Panel and a member of the California Technology Assessment Forum. No other conflicts of interest were reported.