A Self-attention Guided Multi-scale Gradient GAN for Diversified X-ray Image Synthesis

Abstract

Imbalanced image datasets are commonly available in the domain of biomedical image analysis. Biomedical images contain diversified features that are significant in predicting targeted diseases. Generative Adversarial Networks (GANs) are utilized to address the data limitation problem via the generation of synthetic images. Training challenges such as mode collapse, non-convergence, and instability degrade a GAN’s performance in synthesizing diversified and high-quality images. In this work, MSG-SAGAN, an attention-guided multi-scale gradient GAN architecture is proposed to model the relationship between long-range dependencies of biomedical image features and improves the training performance using a flow of multi-scale gradients at multiple resolutions in the layers of generator and discriminator models. The intent is to reduce the impact of mode collapse and stabilize the training of GAN using an attention mechanism with multi-scale gradient learning for diversified X-ray image synthesis. Multi-scale Structural Similarity Index Measure (MS-SSIM) and Frechet Inception Distance (FID) are used to identify the occurrence of mode collapse and evaluate the diversity of synthetic images generated. The proposed architecture is compared with the multi-scale gradient GAN (MSG-GAN) to assess the diversity of generated synthetic images. Results indicate that the MSG-SAGAN outperforms MSG-GAN in synthesizing diversified images as evidenced by the MS-SSIM and FID scores.

A survey on training challenges in generative adversarial networks for biomedical image analysis

“…It shows that the proposed SPGGAN attains better diversity and realistic image synthesis performance than PGGAN yet is distant from real images. Saad et al (2023) also utilized a self-attention mechanism in the multi-scale gradient GAN (MSG-GAN) to generate diversified X-ray images. They integrated a self-attention layer into each layer of the generator and discriminator models.…”

“…The results show better performance of S-CycleGAN regarding training stability. Saad et al (2023) proposed a novel MSG-SAGAN with a relativistic hinge loss function. Relativism in the hinge loss helps the discriminator to improve its learning using approximate predictions of the real images as half of the images are fake on average instead of taking them all as real.…”

“…In this section, state-of-the-art GAN architectures such as DCGAN (Neff et al 2017;Kora Venu and Ravula 2021), AIIN-DCGAN , MSG-GAN (Deepak and Ameer 2020), and MSG-SAGAN (Saad et al 2023) are re-implemented for the COVID-19 chest X-ray image dataset (Rahman et al 2021). The same dataset of X-ray images is used to perform all these experiments.…”

“…The mode collapse problem occurs when the generator produces similar output images while taking different input features. In the domain of biomedical imaging, the mode collapse problem of GANs has been addressed by using minibatch discrimination (Xue et al 2019), skip connections (Segato et al 2020), VAEGAN (Kwon et al 2019), varying layers of generator and discriminator (Qin et al 2020), spectral normalization (Xu et al 2020), perceptual image hashing (Neff et al 2017), Gaussian mixture model as generator (Wu et al 2018b), discriminator with conditional information vector (Modanwal et al 2021), self-attention mechanism (Saad et al 2023;Abdelhalim et al 2021), and adaptive input-image normalization . The non-convergence problem occurs due to the lack of GAN's ability to reach Nash equilibrium.…”

“…The instability problem of GANs occurs due to the vanishing gradient problem. The Wasserstein loss (Xue et al 2019;Segato et al 2020;Kwon et al 2019;Deepak and Ameer 2020), residual connections (Wei et al 2020), multi-scale generator (Wu et al 2018a), and Relativistic hinge loss (Saad et al 2023) techniques are identified to address the instability problem in the biomedical imagery.…”