GAN-based synthetic brain MR image generation

Han, Changhee; Hayashi, H.; Rundo, Leonardo; Araki, Ryosuke; Shimoda, Wataru; Muramatsu, Shogo; Furukawa, Yukio; Mauri, Giancarlo; Nakayama, Hideki

doi:10.1109/isbi.2018.8363678

Cited by 287 publications

(165 citation statements)

References 14 publications

Supporting

Mentioning

162

Contrasting

Unclassified

Order By: Relevance

“…Few recent studies have independently proposed GAN models for multi-contrast MRI synthesis [62], [73], [74]. Perhaps, the closest to our approach are [62] and [73] where conditional GANs with pixel-wise loss were used for improved segmentation based on synthesized FLAIR, T 1 -and T 2 -weighted images.…”

Section: Discussionmentioning

confidence: 99%

Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks

Dar

Yurt

Karacan

et al. 2019

IEEE Trans. Med. Imaging

424

281

View full text Add to dashboard Cite

Acquiring images of the same anatomy with multiple different contrasts increases the diversity of diagnostic information available in an MR exam. Yet, the scan time limitations may prohibit the acquisition of certain contrasts, and some contrasts may be corrupted by noise and artifacts. In such cases, the ability to synthesize unacquired or corrupted contrasts can improve diagnostic utility. For multi-contrast synthesis, the current methods learn a nonlinear intensity transformation between the source and target images, either via nonlinear regression or deterministic neural networks. These methods can, in turn, suffer from the loss of structural details in synthesized images. Here, in this paper, we propose a new approach for multi-contrast MRI synthesis based on conditional generative adversarial networks. The proposed approach preserves intermediate-to-high frequency details via an adversarial loss, and it offers enhanced synthesis performance via pixel-wise and perceptual losses for registered multi-contrast images and a cycle-consistency loss for unregistered images. Information from neighboring cross-sections are utilized to further improve synthesis quality. Demonstrations on T 1-and T 2-weighted images from healthy subjects and patients clearly indicate the superior performance of the proposed approach compared to the previous state-of-the-art methods. Our synthesis approach can help improve the quality and versatility Manuscript

show abstract

Section: Discussionmentioning

confidence: 99%

Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks

Dar

Yurt

Karacan

et al. 2019

IEEE Trans. Med. Imaging

424

281

View full text Add to dashboard Cite

show abstract

“…Because the field is moving so fast, it needs to be pointed out that unlike in 3D GANs, using, for example, convolutional deep belief networks (Wu et al, 2015), we are dealing with fully texturized 3D image stacks and not only 3D binary shapes. Also, although the use of GANs in biomedical imaging is rapidly advancing, for example, for synthesizing artificial brain magnetic resonance images (Han et al, 2018;Kazuhiro et al, 2018) or thyroid tissue imaged by optical coherence tomography (Zhang et al, 2018), 3D applications like GANs for segmentation of liver CT scans (Yang et al, 2017) are still rare. Creating synthetic 3D test data, above all, demands accurately segmented and validated ground truth data for training.…”

Section: Discussionmentioning

confidence: 99%

Generative Adversarial Networks for Augmenting Training Data of Microscopic Cell Images

Baniukiewicz

Lutton

Collier

et al. 2019

Front. Comput. Sci.

View full text Add to dashboard Cite

Generative adversarial networks (GANs) have recently been successfully used to create realistic synthetic microscopy cell images in 2D and predict intermediate cell stages. In the current paper we highlight that GANs can not only be used for creating synthetic cell images optimized for different fluorescent molecular labels, but that by using GANs for augmentation of training data involving scaling or other transformations the inherent length scale of biological structures is retained. In addition, GANs make it possible to create synthetic cells with specific shape features, which can be used, for example, to validate different methods for feature extraction. Here, we apply GANs to create 2D distributions of fluorescent markers for F-actin in the cell cortex of Dictyostelium cells (ABD), a membrane receptor (cAR1), and a cortex-membrane linker protein (TalA). The recent more widespread use of 3D lightsheet microscopy, where obtaining sufficient training data is considerably more difficult than in 2D, creates significant demand for novel approaches to data augmentation. We show that it is possible to directly generate synthetic 3D cell images using GANs, but limitations are excessive training times, dependence on high-quality segmentations of 3D images, and that the number of z-slices cannot be freely adjusted without retraining the network. We demonstrate that in the case of molecular labels that are highly correlated with cell shape, like F-actin in our example, 2D GANs can be used efficiently to create pseudo-3D synthetic cell data from individually generated 2D slices. Because high quality segmented 2D cell data are more readily available, this is an attractive alternative to using less efficient 3D networks.

show abstract

“…Generative models have also been successful at simulating images, either from electronic microscopy (Han et al , 2018b), fluorescence microscopy (Goldsborough et al , ; Osokin et al , ; Lafarge et al , ), or brain magnetic resonance (Han et al , 2018a). Lafarge et al () developed VAE+, a deep generative model for fluorescence microscopy cell images based on a VAE learned with an adversarial mechanism.…”

Section: Applications To Molecular Biology and Biomedical Researchmentioning

confidence: 99%

Enhancing scientific discoveries in molecular biology with deep generative models

Lopez

Gayoso

Yosef

2020

Molecular Systems Biology

View full text Add to dashboard Cite

Generative models provide a well‐established statistical framework for evaluating uncertainty and deriving conclusions from large data sets especially in the presence of noise, sparsity, and bias. Initially developed for computer vision and natural language processing, these models have been shown to effectively summarize the complexity that underlies many types of data and enable a range of applications including supervised learning tasks, such as assigning labels to images; unsupervised learning tasks, such as dimensionality reduction; and out‐of‐sample generation, such as de novo image synthesis. With this early success, the power of generative models is now being increasingly leveraged in molecular biology, with applications ranging from designing new molecules with properties of interest to identifying deleterious mutations in our genomes and to dissecting transcriptional variability between single cells. In this review, we provide a brief overview of the technical notions behind generative models and their implementation with deep learning techniques. We then describe several different ways in which these models can be utilized in practice, using several recent applications in molecular biology as examples.

show abstract

GAN-based synthetic brain MR image generation

Cited by 287 publications

References 14 publications

Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks

Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks

Generative Adversarial Networks for Augmenting Training Data of Microscopic Cell Images

Enhancing scientific discoveries in molecular biology with deep generative models

Contact Info

Product

Resources

About