No cross-validation
Commenting on the findings for this news organization, Vince Calhoun, PhD, neuroscientist and founding director of the Center for Translational Research in Neuroimaging and Data Science, Atlanta, a joint effort between Georgia State, Georgia Tech, and Emory University, noted some study limitations.
One cautionary note is that the investigators “appear to select relevant regions to include in the model based on activation to the task, then computed the predictions using the subset of regions that showed strong activation. The issue is this was done on the same data, so there was no cross-validation of this ‘feature selection’ step,” said Dr. Calhoun, who was not involved with the research. “This is a type of circularity which can lead to inflated accuracies,” he added.
Dr. Calhoun also noted that “multiple ADHD classification studies” have reported accuracies above 90%. In addition, there were only 80 participants in the current dataset.
“That’s relatively small for making strong claims about high accuracies as has been reported elsewhere,” he said.
Dr. McNorgan and Dr. Calhoun have reported no relevant financial relationships.
A version of this article first appeared on Medscape.com.