Only One Dermatology App Cleared By the FDA
Currently, the only FDA-cleared app for dermatologic use is the DermaSensor, an AI-driven device. It was cleared for use in January 2024 for the evaluation of skin lesions “suggestive” of melanoma, basal cell carcinoma, and/or squamous cell carcinoma in patients aged ≥ 40 years “to assist health care providers in determining whether to refer a patient to a dermatologist,” according to an FDA announcement.
Using elastic scattering spectroscopy to analyze light reflecting off the skin to detect malignancy, the manufacturer’s promotional material claims a 96% sensitivity and a 97% specificity.
While Dr. Rotemberg did not comment on these claims, she cautioned that AI models differ with regards to how they were trained and the relative heterogeneity of the training dataset defined by types of patients, types of skin, and types of AI learning processes. All of these variables are relevant in whether the AI will perform in a given clinical setting at the level it performed during development.
“The most accurate models employ narrow datasets, but these do not necessarily mimic what we see in practice,” she said.
In addition, even when an AI-based system is working for a given task, it must be monitored over time. Dr. Rotemberg warned about the potential for “data drift,” which describes the slow evolution in how diseases present, their prevalence by age, or other factors that might affect AI performance. She explained that repeated validation is needed to ensure that the AI-based models remain as accurate over time as they were when first used.
Many of these concepts were explored in a consensus statement from the International Skin Imaging Collaboration AI Working Group, published in JAMA Dermatology in December 2021. The statement, of which Dr. Rotemberg was a coauthor, provided recommendations for the principles of AI algorithm development specific to dermatologic considerations.
At the AAD symposium, Dr. Rotemberg asked the audience for suggestions about the needs they hoped AI might address for in office care or efficiency. Their responses included generating prior authorizations for prescriptions, triaging email for importance, and helping to improve efficiency for common front desk tasks. She liked all of these suggestions, but she warned that as powerful as it can be, AI is not likely to provide technology that will fit seamlessly into workflows without adjustment.
“Our current systems do not allow human integration of AI models,” Dr. Rotemberg said. Rather than counting on AI to adapt to current practices, she cautioned that “we may have to redesign our entire structure to actually be able to accommodate AI-based” systems. The risk for users is tasks that become more challenging before they become easier.
AI Should Not Be a Black Box
AI is promising, but it is not magic, according to other investigators, including Tofunmi A. Omiye, PhD, a postdoctoral scholar in dermatology at Stanford University, California. First author of a recent review of AI in dermatology published in Frontiers in Medicine, Dr. Omiye agreed that clinicians who want to employ AI should be able to understand basic principles if they want the technology to perform as expected.
“I totally agree that physicians should at least have a basic understanding of the data sources for training AI models as we have found that to be important to the performance of these models in the clinical setting,” he told this news organization.
“Beyond understanding the data sources, I believe physicians can also try to have a comprehensive understanding of what AI means, its training process, and evaluation as this will help them to evaluate its utility in their practice,” he added. He also reinforced the relevance of data drift.
“Concepts like distribution shift — where models perform less well over time due to changes in the patient population — are also important to keep in mind,” Dr. Omiye said.
Dr. Wongvibulsin, Dr. Rotemberg, and Dr. Omiye reported no potential financial conflicts of interest relevant to this topic.
A version of this article appeared on Medscape.com .