Image Embedding and Model Ensembling for Automated Chest X-Ray Interpretation

Abstract

Chest X-ray (CXR) is perhaps the most frequentlyperformed radiological investigation globally. In this work, we present and study several machine learning approaches to develop automated CXR diagnostic models. In particular, we trained several Convolutional Neural Networks (CNN) on the CheXpert dataset, a large collection of more than 200k CXR labeled images. Then, we used the trained CNNs to compute embeddings of the CXR images, in order to train two sets of tree-based classifiers from them. Finally, we described and compared three ensembling strategies to combine together the classifiers trained. Rather than expecting some performancewise benefits, our goal in this work is showing that the above two methodologies, i.e., the extraction of image embeddings and models ensembling, can be effective and viable to solve tasks that require medical imaging understanding. Our results in that perspective are encouraging and worthy of further investigation.

Image Embeddings Extracted from CNNs Outperform Other Transfer Learning Approaches in Classification of Chest Radiographs

“…For multiple labels, we selected the maximum output probability of the network for CheXpert labels as the predicted value for the respective HUM-CXR outcome. As reported in previous works [ 15 , 23 ], none of the trained CNNs outperformed any of the other networks on the label problem. Thus, to improve the overall classification performances, we combined the outputs of the trained CNSs through two ensemble methods: simple average and entropy-weighted average.…”

“…Recently, Pham et al [ 23 ] improved state-of-the-art results on CheXpert, proposing an ensemble of CNN architectures. We used the same dataset as Irvin et al [ 37 ], Pham et al [ 23 ], and Giacomello et al [ 15 ] for pretraining; however, whereas they focused on only five representative findings, we enlarged the classification to seven classes. We can compare the performance of cardiomegaly and pleural effusion, the two findings that are most similar between HUM-CXRs and CheXpert.…”

“…For both datasets, we selected only anteroposterior images. Concerning CheXpert, following the approach described in [ 15 ], we resized the images to 256 × 256, and a chest region of 224 × 224 was extracted using a template-matching algorithm. We then normalized the images by scaling their values in the range [0, 1]; because the original models were pretrained on ImageNet, we further standardized them with respect to ImageNet mean and standard deviation.…”

“…In this work, we trained several classifiers on the CheXpert dataset to predict CXR findings. Following the protocol described in [ 15 ], we considered seven convolutional neural networks (CNNs) with different topologies and numbers of parameters: DenseNet121 (7M parameters), DenseNet169 (12,5M parameters), DenseNet201 (18M parameters) [ 16 ], InceptionResNetV2 (54M parameters) [ 17 ], Xception (21M parameters) [ 18 ], VGG16 (15M parameters) [ 19 ], and VGG19 (20M parameters) [ 19 ]. We selected these seven network architectures because (i) they are the most common architectures used to perform classification, and (ii) the performance of each architecture differed depending on the labels.…”

“…These seven networks were not trained from scratch; instead, following a common practice in CNN training, we performed a first transfer learning step by initializing the convolutional layers of the networks with weights of pretrained models on the ImageNet dataset [ 21 ]. Then, we trained all the weights (both convolutional and classification layers) on the CheXpert dataset, using 90% of the sample for training and 10% for validation (further details on the training process can be found in our previous work [ 15 ]).…”