Enhancing Automated COVID-19 Chest X-ray Diagnosis by Image-to-Image GAN Translation

Abstract

The severe pneumonia induced by the infection of the SARS-CoV-2 virus causes massive death in the ongoing COVID-19 pandemic. The early detection of the SARS-CoV-2 induced pneumonia relies on the unique patterns of the chest X-Ray images. Deep learning is a data-greedy algorithm to achieve high performance when adequately trained. A common challenge for machine learning in the medical domain is the accessibility to properly annotated data. In this study, we apply a conditional adversarial network (cGAN) to perform image to image (Pix2Pix) translation from the non-COVID-19 chest X-Ray domain to the COVID-19 chest X-Ray domain. The objective is to learn a mapping from the normal chest X-Ray visual patterns to the COVID-19 pneumonia chest X-ray patterns. The original dataset has a typical imbalanced issue because it contains only 219 COVID-19 positive images but has 1,341 images for normal chest X-Ray and 1,345 images for viral pneumonia. A U-Net based architecture is applied for the image-to-image translation to generate synthesized COVID-19 X-Ray chest images from the normal chest X-ray images. A 50-convolutional-layer residual net (ResNet) architecture is applied for the final classification task. After training the GAN model for 100 epochs, we use the GAN generator to translate 1,100 COVID-19 images from the normal X-Ray to form a balanced training dataset (3,762 images) for the classification task. The ResNet based classifier trained by the enhanced dataset achieves the classification accuracy of 97.8% compared to 96.1% in the transfer learning mode. When trained with the original imbalanced dataset, the model achieves an accuracy of 96.1% compared to 95.6% in the training from trainby-scratch model. In addition, the classifier trained by the enhanced dataset has more stable measures in precision, recall, and F1 scores across different image classes. We conclude that the GAN-based data enhancement strategy is applicable to most medical image pattern recognition tasks, and it provides an effective way to solve the common expertise dependence issue in the medical domain.

Combating COVID-19 Using Generative Adversarial Networks and Artificial Intelligence for Medical Images: Scoping Review

“…As shown in Table 2 , the included studies have reported 5 different tasks being addressed: augmentation (data augmentation), diagnosis of COVID-19, prognosis, segmentation (to identify the lung region), and diagnosis of lung diseases. As the diagnosis of COVID-19 using medical imaging has been a priority since the pandemic started, 39 (68%) of 57 studies reported the diagnosis of COVID-19 as the main focus of their work [ 19 - 21 , 23 - 33 , 35 - 37 , 39 , 41 , 42 , 44 , 46 , 50 , 52 , 53 , 55 , 56 , 58 - 60 , 63 - 69 , 71 , 72 ]. In addition, 9 (16%) studies reported data augmentation as the main task addressed in the work [ 18 , 43 , 45 , 49 , 54 , 61 , 62 ], 1 (2%) study reported prognosis of COVID-19 [ 22 ], 3 (5%) studies reported segmentation of lungs [ 34 , 51 , 57 ], and 1 (2%) study reported diagnosis of multiple lung diseases [ 47 ].…”

“…The cycleGAN is an image translation GAN that does not require paired data to transform images from one domain to another. Other popular types of GANs were conditional GAN used by 9 (16%) studies [ 18 , 22 , 24 , 25 , 33 , 37 , 41 , 57 , 60 ], deep convolutional GAN used by 4 (7%) studies [ 21 , 38 , 43 , 67 ], and auxiliary classifier GAN used by 4 (7%) studies [ 32 , 40 , 55 , 69 ]. The superresolution GAN was used by 2 (4%) studies [ 44 , 68 ], and 1 (2%) study reported the use of multiple GANs, namely Wassertein GAN, auxiliary classifier GAN, and deep convolutional GAN, and compared their performances for improving the quality of images [ 31 ].…”

A Review Study of the Deep Learning Techniques used for the Classification of Chest Radiological Images for COVID-19 Diagnosis

“…To mitigate the consequences of insufficient data, some researchers were working to expand the data set with more comprehensive and effective images ( ( Misztal et al, 2020 ) ( Wang et al, 2021 )), and some used data augmentation methods to add simulated or processed images to mimic the real images ( ( Elzeki et al, 2021 ) ( Ismael and Şengür, 2021 ) ( Liang et al, 2020 ) ( Menon et al, 2020 )). Generative Adversarial Networks (GANs) is popular for data augmentation and in this phase, some GAN optimization ideas were successfully achieved.…”

“…Generative Adversarial Networks (GANs) is popular for data augmentation and in this phase, some GAN optimization ideas were successfully achieved. Liang et al ( Liang et al, 2020 ) applied conditional generative adversarial networks (cGAN) that involved the conditional generation of images. In this architecture the U-net was used for both the Generator (G) network and the Decoder (D) network to make the architecture simpler.…”

Sub-Image Histogram Equalization using Coot Optimization Algorithm for Segmentation and Parameter Selection

“…Hence, higher contrast makes the image regions more separable and it is important to enhance the distorted contrast [1]. Contrast enhancement is significant in the field of computer vision and used for several applications such as retinal image enhancement [2], underwater image enhancement [3] and chest x-ray enhancement [4]. Researchers proposed many techniques for enhancing the contrast of images.…”