Simulation and Synthesis in Medical Imaging

Dou

IEEE Trans. Med. Imaging

et al. 2020

331

185

Unsupervised domain adaptation has increasingly gained interest in medical image computing, aiming to tackle the performance degradation of deep neural networks when being deployed to unseen data with heterogeneous characteristics. In this work, we present a novel unsupervised domain adaptation framework, named as Synergistic Image and Feature Alignment (SIFA), to effectively adapt a segmentation network to an unlabeled target domain. Our proposed SIFA conducts synergistic alignment of domains from both image and feature perspectives. In particular, we simultaneously transform the appearance of images across domains and enhance domain-invariance of the extracted features by leveraging adversarial learning in multiple aspects and with a deeply supervised mechanism. The feature encoder is shared between both adaptive perspectives to leverage their mutual benefits via end-to-end learning. We have extensively evaluated our method with cardiac substructure segmentation and abdominal multi-organ segmentation for bidirectional crossmodality adaptation between MRI and CT images. Experimental results on two different tasks demonstrate that our SIFA method is effective in improving segmentation performance on unlabeled target images, and outperforms the state-of-the-art domain adaptation approaches by a large margin.

Section: Discussionmentioning

confidence: 99%

Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation

Dou

IEEE Trans. Med. Imaging

et al. 2020

331

185

Lecture Notes in Computer Science

“…Medical image simulation and synthesis have been studied for a while and are increasingly getting traction in medical imaging community [7]. It is partly due to the exponential growth in data availability, and partly due to the availability of better machine learning models and supporting systems.…”

Section: Introductionmentioning

confidence: 99%

“…It is partly due to the exponential growth in data availability, and partly due to the availability of better machine learning models and supporting systems. Twelve recent research on medical image synthesis and simulation were presented in the special issue of Simulation and Synthesis in Medical Imaging [7].…”

Section: Introductionmentioning

confidence: 99%

Medical Image Synthesis for Data Augmentation and Anonymization Using Generative Adversarial Networks

Shin

Tenenholtz

Rogers

et al. 2018

475

305

Data diversity is critical to success when training deep learning models. Medical imaging data sets are often imbalanced as pathologic findings are generally rare, which introduces significant challenges when training deep learning models. In this work, we propose a method to generate synthetic abnormal MRI images with brain tumors by training a generative adversarial network using two publicly available data sets of brain MRI. We demonstrate two unique benefits that the synthetic images provide. First, we illustrate improved performance on tumor segmentation by leveraging the synthetic images as a form of data augmentation. Second, we demonstrate the value of generative models as an anonymization tool, achieving comparable tumor segmentation results when trained on the synthetic data versus when trained on real subject data. Together, these results offer a potential solution to two of the largest challenges facing machine learning in medical imaging, namely the small incidence of pathological findings, and the restrictions around sharing of patient data.

IEEE Trans. Med. Imaging

“…Medical image synthesis is defined as the generation of realistic images through learning models [1]. From a technical perspective, image synthesis can be achieved from a generative model (e.g., from noise) or a cross-modality adaptation model (e.g., from MRI to CT).…”

Section: A Cross-modality Image Synthesismentioning

confidence: 99%

“…MRI to CT, CT to MRI etc.) [1]. While there are impediments to use the synthetic images directly in clinical practice, synthetic images have been shown to be an effective intermediate This research was supported by NSF CAREER 1452485 (Landman), NIH grants 5R21EY024036 (Landman), R01EB017230 (Landman), 1R21NS064534 (Prince/Landman), 1R01NS070906 (Pham), 2R01EB006136 (Dawant), 1R03EB012461 (Landman), NCI Cancer Center Support Grant (P30 CA068485), and R01NS095291 (Dawant).…”

Section: Introductionmentioning

confidence: 99%

SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truth

Huo

Moon

et al. 2019

229

124

A key limitation of deep convolutional neural networks (DCNN) based image segmentation methods is the lack of generalizability. Manually traced training images are typically required when segmenting organs in a new imaging modality or from distinct disease cohort. The manual efforts can be alleviated if the manually traced images in one imaging modality (e.g., MRI) are able to train a segmentation network for another imaging modality (e.g., CT). In this paper, we propose an end-to-end synthetic segmentation network (SynSeg-Net) to train a segmentation network for a target imaging modality without having manual labels. SynSeg-Net is trained by using (1) unpaired intensity images from source and target modalities, and (2) manual labels only from source modality. SynSeg-Net is enabled by the recent advances of cycle generative adversarial networks (CycleGAN) and DCNN. We evaluate the performance of the SynSeg-Net on two experiments: (1) MRI to CT splenomegaly synthetic segmentation for abdominal images, and (2) CT to MRI total intracranial volume synthetic segmentation (TICV) for brain images. The proposed end-to-end approach achieved superior performance to two stage methods. Moreover, the SynSeg-Net achieved comparable performance to the traditional segmentation network using target modality labels in certain scenarios. The source code of SynSeg-Net is publicly available 2 .