Attention-based Generative Adversarial Network in Medical Imaging: A Review (Preprint)

Abstract

BACKGROUND As a popular probabilistic generative model, generative adversarial network (GAN) has been successfully used not only in natural image processing, but also in medical image analysis and computer-aided diagnosis. Despite the various advantages, the applications of GAN in medical image analysis face new challenges. The introduction of attention mechanisms, which resembles the human visual system that focuses on the task-related local image area for certain information extraction, has drawn increasing interest. Recently proposed transformer-based architectures that leverage self-attention mechanism encode long-range dependencies and learn representations that are highly expressive. This motivates us to summarize the applications of using transformer-based GAN for medical image analysis. OBJECTIVE This review aimed to survey both GAN and attention mechanisms with particular application for medical image analysis. This review also provided a brief introduction of some well-known GAN variations and attention mechanisms. METHODS To organize this review comprehensively, we conducted a literature search in PubMed, arXiv, Society of Photo-Optical Instrumentation Engineer Medical Imaging (SPIE), IEEE International Symposium on Biomedical Imaging (ISBI), and International Conference on Deep Learning for Medical Imaging (MIDL). We set the search deadline to March 30, 2022. RESULTS We reviewed recent advances in techniques combining various attention modules with different adversarial training schemes, and their applications in medical segmentation, synthesis and detection. Several recent studies have shown that attention modules can be effectively incorporated into a GAN model in detecting lesion areas and extracting diagnosis-related feature information precisely, thus providing a useful tool for medical image processing and diagnosis. CONCLUSIONS This review indicates that research on the medical imaging analysis of GAN and attention mechanisms is still at an early stage despite the great potential. We highlight the attention-based generative adversarial network is an efficient and promising computational model advancing future research and applications in medical image analysis.

BESRGAN: Boundary equilibrium face super‐resolution generative adversarial networks

“…Based on the theoretical flaws of GANs, BEGAN replaced a classifier with an auto‐encoder as a discriminator [48], and matched the auto‐encoder loss distributions rather than the data distributions. In BEGAN, the generator and the discriminator are trained simultaneously with an equilibrium constraint, so there is more stable training than the regular GANs.…”

Synthetic data generator: understanding human face & body via image synthesis

“…Generally speaking, self-attention, is an attention mechanism that captures dependencies at different positions of a single sequence without recurrent calculations. Recently, it has been shown to be very useful in computer vision tasks such as image classification [66,67], image generation [68,69], and scene segmentation [68,70]. Different from existing work, our DA-GAN employs two different types of attention to jointly capture long-range dependencies and local features.…”