Chest Radiographs in Congestive Heart Failure: Visualizing Neural Network Learning

Abstract

To examine Generative Visual Rationales (GVRs) as a tool for visualizing neural network learning of chest radiograph features in congestive heart failure (CHF). Materials and Methods: A total of 103 489 frontal chest radiographs in 46 712 patients acquired from January 1, 2007, to December 31, 2016, were divided into a labeled data set (with B-type natriuretic peptide [BNP] result as a marker of CHF) and unlabeled data set (without BNP result). A generative model was trained on the unlabeled data set, and a neural network was trained on the encoded representations of the labeled data set to estimate BNP. The model was used to visualize how a radiograph with high estimated BNP would look without disease (a "healthy" radiograph). An overfitted model was developed for comparison, and 100 GVRs were blindly assessed by two experts for features of CHF. Area under the receiver operating characteristic curve (AUC), k coefficient, and mixed-effects logistic regression were used for statistical analyses. Results: At a cutoff BNP of 100 ng/L as a marker of CHF, the correctly trained model achieved an AUC of 0.82. Assessment of GVRs revealed that the correctly trained model highlighted conventional radiographic features of CHF as reasons for an elevated BNP prediction more frequently than the overfitted model, including cardiomegaly (153 [76.5%] of 200 vs 64 [32%] of 200, respectively; P , .001) and pleural effusions (47 [23.5%] of 200 vs 16 [8%] of 200, respectively; P = .003). Conclusion: Features of congestive heart failure on chest radiographs learned by neural networks can be identified using Generative Visual Rationales, enabling detection of bias and overfitted models.

Identifying cardiomegaly in chest X-rays: a cross-sectional study of evaluation and comparison between different transfer learning methods

“…Previous studies have used other deep learning methods, with variable results in detecting cardiomegaly but also using a smaller sample size (36)(37)(38)(39). Another study has tested 3 transfer learning algorithms including Inception 3 but found out much lower accuracy levels than the ones we describe here (40).…”

Breaking Medical Data Sharing Boundaries by Employing Artificial Radiographs

“…Moreover, our developed GANs can be used to visualize what the generator neural network sees and to reveal correlations between diseases. This might be used in several ways: as a check that the GAN has been trained correctly, as a tool to uncover relationships between diseases or to visualize hallmark changes of pathologies [37] and potentially also as a decision support system for diagnosis. Another advantage of the proposed concept of data sharing is the immense reduction in data storage requirements: storing of a single radiograph image with a resolution of 1024 × 1024 and a bit depth of 8 bits requires 1 megabyte of hard drive space.…”

Machine learning in cardiovascular magnetic resonance: basic concepts and applications

“…Another limitation is the ‘black-box’ nature of DL algorithms since it is often unclear what information is used to come to a certain classification or result. Techniques to visualize salient features can potentially help address this limitation [88]. Furthermore, the present lack of model robustness and lack of portability with respect to different CMR scanners, sequences, imaging parameters and institutions need to be addressed.…”