MRI Cross-Modality Image-to-Image Translation

Abstract

We present a cross-modality generation framework that learns to generate translated modalities from given modalities in MR images. Our proposed method performs Image Modality Translation (abbreviated as IMT) by means of a deep learning model that leverages conditional generative adversarial networks (cGANs). Our framework jointly exploits the low-level features (pixel-wise information) and high-level representations (e.g. brain tumors, brain structure like gray matter, etc.) between cross modalities which are important for resolving the challenging complexity in brain structures. Our framework can serve as an auxiliary method in medical use and has great application potential. Based on our proposed framework, we first propose a method for cross-modality registration by fusing the deformation fields to adopt the cross-modality information from translated modalities. Second, we propose an approach for MRI segmentation, translated multichannel segmentation (TMS), where given modalities, along with translated modalities, are segmented by fully convolutional networks (FCN) in a multichannel manner. Both of these two methods successfully adopt the crossmodality information to improve the performance without adding any extra data. Experiments demonstrate that our proposed framework advances the state-of-the-art on five brain MRI datasets. We also observe encouraging results in cross-modality registration and segmentation on some widely adopted brain datasets. Overall, our work can serve as an auxiliary method in medical use and be applied to various tasks in medical fields.

Enhancing magnetic resonance imaging-driven Alzheimer’s disease classification performance using generative adversarial learning

“…Since its introduction, there has been a surge of interest in the application of GAN frameworks related to the brain. Some of the applications include image generation with improved properties such as achieving super resolution or better quality [6][7][8][9][10][11], data augmentation [12][13][14], segmentation [9,[13][14][15][16], image reconstruction [17][18][19][20], image-to-image translation [21][22][23][24], and motion correction [25,26]. While these important studies have demonstrated the exciting prospect of using GAN architectures, there is a limited amount of work that has focused on utilizing the generated images for subsequent tasks such as disease classification [27].…”

Oncoimmunology Meets Organs-on-Chip

“…Top right orange box displays the key formula associated to the step length and angle variation computations. (Nie et al, 2018;Li et al, 2019c;Xue et al, 2020;Yang et al, 2020), and computational analysis of natural environments (Saleemi et al, 2009;Negin and Brémond, 2019;Comes et al, 2020a,b). GAN are based on a game scenario where the generator network must compete with an adversary, the discriminator network.…”

“…GANs can have disparate practices, covering social sciences, image analysis and biological problem solving. To this purpose, GANs have been applied for prediction of human actions ( Liu et al, 2017 ), image processing applications ( Nie et al, 2018 ; Li et al, 2019c ; Xue et al, 2020 ; Yang et al, 2020 ), and computational analysis of natural environments ( Saleemi et al, 2009 ; Negin and Brémond, 2019 ; Comes et al, 2020a , b ). GAN are based on a game scenario where the generator network must compete with an adversary, the discriminator network.…”

T1-weighted and T2-weighted MRI image synthesis with convolutional generative adversarial networks

“…Moreover, Yang et al proposed the cross-modality generation framework in MRI with GAN [ 11 ]. They resized the MRI images to a resolution of 256 × 256 pixels.…”