The Need for Targeted Labeling of Machine Learning–Based Software as a Medical Device

Abstract

Machine learning (ML)-based clinical decision support (CDS) tools are increasingly part of the health care landscape. These tools have the potential to automatically identify patterns and assign health risk in ways that human medical practitioners are not capable of doing. If implemented correctly, this has the potential to provide a more optimized and less expensive health care. However, there is also a downside where end users-who do not necessarily understand how the underlying ML algorithms operate-need to trust the accuracy of such tools. As the recent paper by Hsu et al 1 reported, sometimes this trust may be unwarranted.ML models for medical imaging has been one of the success stories of ML algorithms. 2 Medical images consist of complex patterns. These patterns can be difficult for humans to decipher; however, ML models can be trained to disentangle them. Imaging AI models have been successfully applied in

Decision region analysis for generalizability of artificial intelligence models: estimating model generalizability in the case of cross-reactivity and population shift

“…This is a dilemma considering the potential advantages that AI technologies 5 have to offer in improving health care delivery. The need for methods of assessing model generalizability is becoming even more important with the advent of evolving algorithms that may be piece-wise modified by the device developer, 6 potentially updated continuously 7 or tailored to a local population 8 – 10 or specific subgroups 11 . Thus, there is a need to innovate and help overcome some of the generalizability assessment limitations in AI-enabled medical devices.…”