“Shortcuts” Causing Bias in Radiology Artificial Intelligence: Causes, Evaluation, and Mitigation

Banerjee, Imon; Bhattacharjee, Kamanasish; Burns, James MacGregor; Trivedi, Hari; Purkayastha, Saptarshi; Seyyed-Kalantari, Laleh; Patel, Bhavik N.; Shiradkar, Rakesh; Gichoya, Judy Wawira

doi:10.1016/j.jacr.2023.06.025

Cited by 24 publications

(9 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The SIENNA AI architecture is developed along with strategies for minimal DICOM data preprocessing for comparability (Fig1) and avoids overprocessing of MRI data that is present in current public datasets and which limits generalizability to the clinic (Fig. 1a) [64]. SIENNA incorporates a robust noninterdependent three-class multi-classification to discriminate between healthy tissue, or GBM or MET tumor pathology (Fig.…”

Section: Resultsmentioning

confidence: 99%

“…Our de novo development of SIENNA ML architecture in this study was motivated by our inability to sufficiently generalize the SCENIC architecture to achieve similar high accuracy analysis of minimally processed MRI DICOM clinical images. The SIENNA AI architecture is developed along with strategies for minimal DICOM data preprocessing for comparability ( Fig1 ) and avoids overprocessing of MRI data that is present in current public datasets and which limits generalizability to the clinic ( Fig.1a ) [64]. SIENNA incorporates a robust non-interdependent three-class multi-classification to discriminate between healthy tissue, or GBM or MET tumor pathology ( Fig.1b ).…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

SIENNA: Generalizable Lightweight Machine Learning Platform for Brain Tumor Diagnostics

Sunil,

Rajeev,

Chatterjee

et al. 2024

Preprint

View full text Add to dashboard Cite

The transformative integration of Machine Learning (ML) for Artificial General Intelligence (AGI)-enhanced clinical imaging diagnostics, is itself in development. In brain tumor pathologies, magnetic resonance imaging (MRI) is a critical step that impacts the decision for invasive surgery, yet expert MRI tumor typing is inconsistent and misdiagnosis can reach levels as high as 85%. Current state-of-the-art (SOTA) ML brain tumor models struggle with data overfitting and susceptibility to shortcut learning, further exacerbated in large-sized models with many tunable parameters. In a comparison with multiple SOTA models, our deep ML brain tumor diagnostics model, SIENNA, surpassed limitations in four key areas of prioritized minimal data preprocessing, an optimized architecture that reduces shortcut learning and overfitting, integrated inductive cross-validation method for generalizability, and smaller neural architecture. SIENNA is applicable across MRI machines and 1.5 and 3.0 Tesla, and achieves high average accuracies on clinical DICOM MRI data across three-way classification: 92% (non-tumor), 91% (GBM), and 93% (MET) with retained high F1 and AUROC values for limited false positives/negatives. SIENNA is a lightweight clinical-ready AGI framework compatible with future multimodal expanded data integration.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

SIENNA: Generalizable Lightweight Machine Learning Platform for Brain Tumor Diagnostics

Sunil,

Rajeev,

Chatterjee

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…With the growth of artificial intelligence (AI) applications in medicine, concern over fairness and transparency has also grown. A growing concern highlighted by multiple studies [1][2][3][4] is the phenomenon of algorithmic shortcutting, wherein DL models grasp superficial correlations in training data, potentially leading to biased or unreliable predictions. This concern holds particular weight in orthopedics, where machine learning is deployed for various applications, ranging from the detection of rare fractures to the classification of injuries and prediction of patient outcomes 5,6 .…”

Section: Introductionmentioning

confidence: 99%

Algorithmic shortcutting in medical image analysis

Koback,

Hill,

Barnum

et al. 2024

Medical Imaging 2024: Computer-Aided Diagnosis

View full text Add to dashboard Cite

As deep learning (DL) gains prominence in medical image analysis, its applications in orthopedics and broader medical contexts have expanded significantly. While machine learning algorithms show potential in using X-ray images for predicting outcomes, biases may arise due to insufficient preprocessing. One such source of bias is algorithmic shortcutting, a phenomenon wherein DL models inadvertently train on patterns within training data unrelated to the intended diagnostic content. Leveraging the Osteoarthritis Initiative (OAI) dataset and ResNet18, our convolutional neural network (CNN) could predict the clinical center from which X-rays originated with over 95% accuracy. Moreover, our investigation unveiled significant variations in patient demographics and outcomes between clinical centers, implying that discerning the originating clinical center could be a potential shortcut in predicting patient outcomes from X-rays. These findings underscore the importance of careful X-ray image preparation and analysis. More broadly, as deep learning models increasingly influence clinical decision-making, this work underscores the importance of addressing model shortcutting to enhance the transparency, interpretability, and efficacy of deep learning research across orthopedics and other medical domains.

show abstract

“… 20 To overcome this confounding, progress must be made to improve model training, evaluation, and explainability. 21 …”

Section: Introductionmentioning

confidence: 99%

AI pitfalls and what not to do: mitigating bias in AI

Gichoya,

Thomas,

Celi

et al. 2023

The British Journal of Radiology

Self Cite

View full text Add to dashboard Cite

Various forms of artificial intelligence (AI) applications are being deployed and used in many healthcare systems. As the use of these applications increases, we are learning the failures of these models and how they can perpetuate bias. With these new lessons, we need to prioritize bias evaluation and mitigation for radiology applications; all the while not ignoring the impact of changes in the larger enterprise AI deployment which may have downstream impact on performance of AI models. In this paper, we provide an updated review of known pitfalls causing AI bias and discuss strategies for mitigating these biases within the context of AI deployment in the larger healthcare enterprise. We describe these pitfalls by framing them in the larger AI lifecycle from problem definition, data set selection and curation, model training and deployment emphasizing that bias exists across a spectrum and is a sequela of a combination of both human and machine factors.

show abstract

“Shortcuts” Causing Bias in Radiology Artificial Intelligence: Causes, Evaluation, and Mitigation

Cited by 24 publications

References 43 publications

SIENNA: Generalizable Lightweight Machine Learning Platform for Brain Tumor Diagnostics

SIENNA: Generalizable Lightweight Machine Learning Platform for Brain Tumor Diagnostics

Algorithmic shortcutting in medical image analysis

AI pitfalls and what not to do: mitigating bias in AI

Contact Info

Product

Resources

About