“We would be surprised if the experimentally designed process changes between these two studies did not result in altered spectra. In fact, a goal of these experiments was to study the spectral alterations produced by changing the process,” they said.
Because serum proteomics is in its infancy, they wrote, there is no procedure to standardize intra- and inter-laboratory comparisons. Only after that standardization happens can well-designed, meaningful, and reproducible studies be conducted.
In the meantime, they concluded, researchers who wish to attempt such studies should keep open lines of communication with those who originally produced the data. “A meaningful analysis of reproducibility requires communication. … Without such communication, data can be misinterpreted; unwarranted, overextended conclusions can be drawn; and misinformation can be spread,” he said.
Additional research using meticulously designed studies is needed, Dr. Baggerly said. “If a test for ovarian cancer ever does come about from these data, I'd need to see a lot more studies before I'd send my mother out to get it.”
OvaCheck Unaffected, Developer Says
The serum proteomics study design debate won't affect the progress of OvaCheck, a proteomics test being developed as a screen for women at high risk for ovarian cancer, said Peter Levine, head of the Maryland firm developing the test.
“This is a purely academic debate,” said Mr. Levine, chief executive officer of Correlogic Systems Inc. “It has no bearing whatsoever on the state of the development of the technology today or on any of the other work researchers have been pursuing in this field.”
OvaCheck uses a sophisticated mathematical algorithm and mass spectroscopy to identify a specific pattern of serum proteins associated with even very early-stage ovarian cancers. The method was based on a 2002 National Cancer Institute (NCI) study, but it uses a different mass spectrometer and different spectral signals to identify cancer samples. Correlogic Systems is conducting validity testing on hundreds of samples but has not released any data on those tests.
The study design debate adds nothing to the development of proteomics technology because it focuses on outdated research, Mr. Levine said. “These studies are 2 and 3 years old,” he said. “Since then, scores of additional papers have been published on this technique and various other techniques.”
In fact, reanalyzing older studies may put forth the mistaken impression that serum proteomics has no future as a screening or diagnostic tool.
“The [NCI study] ushered in a revolution in the way we look at this biological data,” he said. “But it was just a proof-of-concept study. No one ever claimed it was a test for ovarian cancer.”
Many additional, more recent studies continue to expand on this original idea, including the research Correlogic Systems is performing, Mr. Levine said.
“We are refining our own technology as we go through the testing process, and that kind of research and development—tweaking the equipment and the process—goes on forever, as it should. Continuing to debate these early papers is like doing a thesis on the Wright brothers' first flight, when you already have a 747 that flies.”
OvaCheck, however, is still struggling through administrative processes at the Food and Drug Administration. Correlogic Systems hoped to license OvaCheck as a lab-developed test regulated under the Clinical Laboratory Improvement Amendments (CLIA). But the FDA determined last year that the software powering OvaCheck is a medical device covered by interstate commerce regulations and thus subject to FDA premarket review.
“We're still working with FDA on that issue and hope to have it resolved soon,” Mr Levine said.