Image Embeddings Extracted from CNNs Outperform Other Transfer Learning Approaches in Classification of Chest Radiographs

Abstract

To identify the best transfer learning approach for the identification of the most frequent abnormalities on chest radiographs (CXRs), we used embeddings extracted from pretrained convolutional neural networks (CNNs). An explainable AI (XAI) model was applied to interpret black-box model predictions and assess its performance. Seven CNNs were trained on CheXpert. Three transfer learning approaches were thereafter applied to a local dataset. The classification results were ensembled using simple and entropy-weighted averaging. We applied Grad-CAM (an XAI model) to produce a saliency map. Grad-CAM maps were compared to manually extracted regions of interest, and the training time was recorded. The best transfer learning model was that which used image embeddings and random forest with simple averaging, with an average AUC of 0.856. Grad-CAM maps showed that the models focused on specific features of each CXR. CNNs pretrained on a large public dataset of medical images can be exploited as feature extractors for tasks of interest. The extracted image embeddings contain relevant information that can be used to train an additional classifier with satisfactory performance on an independent dataset, demonstrating it to be the optimal transfer learning strategy and overcoming the need for large private datasets, extensive computational resources, and long training times.

Uncovering the effects of model initialization on deep model generalization: A study with adult and pediatric chest X-ray images

“…We are also proposing ensemble methods that merge the weights of multiple models. Our approach is different from traditional techniques that aggregate model predictions [38][39][40][41].…”

Uncovering the effects of model initialization on deep model generalization: A study with adult and pediatric chest X-ray images

“…We are also proposing ensemble methods that merge the weights of multiple models. Our approach is different from traditional techniques that aggregate model predictions [37][38][39]. Our proposed weightlevel ensembles harness the power of diverse weight initializations, capitalizing on complementary learning dynamics to foster robust generalization in complex, high-dimensional medical data landscapes.…”

Editorial on Special Issue “Artificial Intelligence in Image-Based Screening, Diagnostics, and Clinical Care of Cardiopulmonary Diseases”

“…The transfer learning-based approaches are prominently used as they leverage the knowledge learned from a large collection of stock photographic images such as ImageNet [ 16 ] to improve performance and generalization in medical visual recognition tasks with a sparse collection of medical data and their associated labels. In this regard, Gozzi et al [ 17 ] proposed the identification of the optimal transfer learning strategy for a CXR classification task. They followed a systematic procedure which is as follows: (i) Several ImageNet-pretrained CNN models were retrained on the publicly available CheXpert [ 18 ] CXR dataset.…”