Challenges of implementing computer-aided diagnostic models for neuroimages in a clinical setting

Abstract

Advances in artificial intelligence have cultivated a strong interest in developing and validating the clinical utilities of computer-aided diagnostic models. Machine learning for diagnostic neuroimaging has often been applied to detect psychological and neurological disorders, typically on small-scale datasets or data collected in a research setting. With the collection and collation of an ever-growing number of public datasets that researchers can freely access, much work has been done in adapting machine learning models to classify these neuroimages by diseases such as Alzheimer’s, ADHD, autism, bipolar disorder, and so on. These studies often come with the promise of being implemented clinically, but despite intense interest in this topic in the laboratory, limited progress has been made in clinical implementation. In this review, we analyze challenges specific to the clinical implementation of diagnostic AI models for neuroimaging data, looking at the differences between laboratory and clinical settings, the inherent limitations of diagnostic AI, and the different incentives and skill sets between research institutions, technology companies, and hospitals. These complexities need to be recognized in the translation of diagnostic AI for neuroimaging from the laboratory to the clinic.

Cortical thickness and grey-matter volume anomaly detection in individual MRI scans: Comparison of two methods

“…CE-marked and FDA-approved commercial tools for clinical decision support by brain morphometry have meanwhile become available for application in patients with multiple sclerosis and various forms of dementia. Despite formal approval for diagnostic purposes, a deficiency of these tools is that validation, especially in clinical terms, in many cases still is an open topic of research (Pemberton et al, 2021; Mendelson et al, 2023) due to a multitude of factors (Haller et al, 2022; Leming et al, 2023; Hedderich et al, 2023). This is remarkable, since an international survey among practitioners investigating their application of (commercial or scientific) brain morphometry tools has clearly shown that user acceptance is associated with the availability of technical and clinical validation studies (Vernooij et al, 2019).…”

No-code machine learning in radiology: implementation and validation of a platform that allows clinicians to train their own models

“…With industry salaries increasing relative to what healthcare systems can pay, finding staff to fill such roles is becoming more and more of a challenge. [2] Several no-code platforms for training ML models exist from companies like Amazon, Apple, Calrifai, Google, and Microsoft. [9] Some of these platforms have been tested on publicly available medical imaging datasets.…”

“…[1] However, this pace of AI adoption within healthcare has been slow. [2] One cause of this is that hospitals are cost strapped and still reeling from pandemic, and a tiny fraction of FDA-approved AI devices are covered by insurance. [3] A perhaps even bigger and more serious issue is that external validations of AI algorithms often show substantial drops in performance compared to what was originally reported in the FDA submission.…”

“…With industry salaries increasing relative to what healthcare systems can pay, finding staff to fill such roles is becoming more and more of a challenge. [2]…”

Conformal Triage for Medical Imaging AI Deployment

“…They struggle to generalize : they cannot perform as well on new data as on the data used for development. This may be due to differences in demographics—such as age, sex, acuity of presentation, and disease prevalence—as well as technical differences in hardware and protocols from the carefully curated datasets normally used to train the AI [10, 13]. The performance gap may be immediately evident, or drifting conditions may cause it to appear over time [12].…”