Weakly supervised pneumonia localization in chest X‐rays using generative adversarial networks

Abstract

Purpose: Automatic localization of pneumonia on chest X‐rays (CXRs) is highly desirable both as an interpretive aid to the radiologist and for timely diagnosis of the disease. However, pneumonia's amorphous appearance on CXRs and complexity of normal anatomy in the chest present key challenges that hinder accurate localization. Existing studies in this area are either not optimized to preserve spatial information of abnormality or depend on expensive expert‐annotated bounding boxes. We present a novel generative adversarial network (GAN)‐based machine learning approach for this problem, which is weakly supervised (does not require any location annotations), was trained to retain spatial information, and can produce pixel‐wise abnormality maps highlighting regions of abnormality (as opposed to bounding boxes around abnormality). Methods: Our method is based on the Wasserstein GAN framework and, to the best of our knowledge, the first application of GANs to this problem. Specifically, from an abnormal CXR as input, we generated the corresponding pseudo normal CXR image as output. The pseudo normal CXR is the “hypothetical” normal, if the same abnormal CXR were not to have any abnormalities. We surmise that the difference between the pseudo normal and the abnormal CXR highlights the pixels suspected to have pneumonia and hence is our output abnormality map. We trained our algorithm on an “unpaired” data set of abnormal and normal CXRs and did not require any location annotations such as bounding boxes/segmentations of abnormal regions. Furthermore, we incorporated additional prior knowledge/constraints into the model and showed that they help improve localization performance. We validated the model on a data set consisting of 14 184 CXRs from the Radiological Society of North America pneumonia detection challenge. Results: We evaluated our methods by comparing the generated abnormality maps with radiologist annotated bounding boxes using receiver operating characteristic (ROC) analysis, image similarity metrics such as normalized cross‐correlation/mutual information, and abnormality detection rate.We also present visual examples of the abnormality maps, covering various scenarios of abnormality occurrence. Results demonstrate the ability to highlight regions of abnormality with the best method achieving an ROC area under the curve (AUC) of 0.77 and a detection rate of 85%.The GAN tended to perform better as prior knowledge/constraints were incorporated into the model. Conclusions: We presented a novel GAN based approach for localizing pneumonia on CXRs that (1) does not require expensive hand annotated location ground truth; and (2) was trained to produce abnormality maps at the pixel level as opposed to bounding boxes. We demonstrated the efficacy of our methods via quantitative and qualitative results.

Improved Chest Anomaly Localization without Pixel-level Annotation via Image Translation Network Application in Pseudo-paired Registration Domain

“…Weakly supervised GANs based on training with both normal and abnormal labeled data have recently been adopted to further improve performance. In addition to exploiting abnormal data during learning, such GANs outperform those based on unsupervised learning for accurate anomaly localization and stability [18,19,20]. In principle, these methods are based on image translation [21,22], which synthesizes normal CXR images from CXR images containing disease regions.…”