MRI‐only based synthetic CT generation using dense cycle consistent generative adversarial networks

Lei, Yang; Harms, Joseph; Wang, Tonghe; Liu, Yingzi; Shu, Hui‐Kuo; Jani, Ashesh B.; Curran, Walter J.; Mao, Hui; Liu, Tian; Yang, Xiaofeng

doi:10.1002/mp.13617

Cited by 243 publications

(241 citation statements)

References 53 publications

(131 reference statements)

Supporting

Mentioning

235

Contrasting

Order By: Relevance

“…Because local mismatches between MR and CT images remain even after deformable registration, as the images have fundamentally different properties, training a CT‐to‐MRI transformation model is difficult. To cope with this challenge, inspired by a recent 2D CycleGAN study, we introduced a 3D CycleGAN in our sMRI generation algorithm because of its ability to mimic target data distribution by incorporating an inverse MRI‐to‐CT mapping . In addition, the patient anatomy can vary significantly among individuals.…”

Section: Methodsmentioning

confidence: 99%

“…To cope with this challenge, inspired by a recent 2D CycleGAN study, 22 we introduced a 3D CycleGAN in our sMRI generation algorithm because of its ability to mimic target data distribution by incorporating an inverse MRI-to-CT mapping. 26 In addition, the patient anatomy can vary significantly among individuals. In order to accurately predict each voxel in the anatomic region (air, bone, and soft tissue), inspired by densely connected CNN, 27 we introduced several dense blocks to capture multiscale information (including low-frequency and high-frequency) by extracting features from previous and following hidden layers.…”

Section: Synthetic Mri Generationmentioning

confidence: 99%

See 1 more Smart Citation

CT prostate segmentation based on synthetic MRI‐aided deep attention fully convolution network

et al. 2019

Self Cite

View full text Add to dashboard Cite

Purpose Accurate segmentation of the prostate on computed tomography (CT) for treatment planning is challenging due to CT's poor soft tissue contrast. Magnetic resonance imaging (MRI) has been used to aid prostate delineation, but its final accuracy is limited by MRI‐CT registration errors. We developed a deep attention‐based segmentation strategy on CT‐based synthetic MRI (sMRI) to deal with the CT prostate delineation challenge without MRI acquisition. Methods and materials We developed a prostate segmentation strategy which employs an sMRI‐aided deep attention network to accurately segment the prostate on CT. Our method consists of three major steps. First, a cycle generative adversarial network was used to estimate an sMRI from CT images. Second, a deep attention fully convolution network was trained based on sMRI and the prostate contours deformed from MRIs. Attention models were introduced to pay more attention to prostate boundary. The prostate contour for a query patient was obtained by feeding the patient's CT images into the trained sMRI generation model and segmentation model. Results The segmentation technique was validated with a clinical study of 49 patients by leave‐one‐out experiments and validated with an additional 50 patients by hold‐out test. The Dice similarity coefficient, Hausdorff distance, and mean surface distance indices between our segmented and deformed MRI‐defined prostate manual contours were 0.92 ± 0.09, 4.38 ± 4.66, and 0.62 ± 0.89 mm, respectively, with leave‐one‐out experiments, and were 0.91 ± 0.07, 4.57 ± 3.03, and 0.62 ± 0.65 mm, respectively, with hold‐out test. Conclusions We have proposed a novel CT‐only prostate segmentation strategy using CT‐based sMRI, and validated its accuracy against the prostate contours that were manually drawn on MRI images and deformed to CT images. This technique could provide accurate prostate volume for treatment planning without requiring MRI acquisition, greatly facilitating the routine clinical workflow.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Synthetic Mri Generationmentioning

confidence: 99%

CT prostate segmentation based on synthetic MRI‐aided deep attention fully convolution network

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this work, mean absolute error (MAE) and gradient difference error (GDE) were used as a compound loss to calculate the cycle consistency loss of generator [14]. The MAE loss forces the generator to synthesis RSP images with accurate voxel intensity to a level of ground truth RSP images.…”

Section: Loss Functionmentioning

confidence: 99%

“…The physical derivation does not accommodate these non-idealities, and would magnify the noise and artifacts on the DECT images to the derived parametric maps that directly lead to uncertainty and inaccuracy. With the development of machine learning in recent years, novel methods have been developed to convert between images presenting similar anatomy but different modalities [1,3,4,7,11,13,14,15,33,34]. Due to the data-driven properties, learning-based methods are expected to be more robust to noise.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep learning-based relative stopping power mapping generation with cone-beam CT in proton radiation therapy

Wang

Harms

Lei

et al. 2020

Medical Imaging 2020: Physics of Medical Imaging

Self Cite

View full text Add to dashboard Cite

Purpose:Dual-energy CT (DECT) has been used to derive relative stopping power (RSP) map by obtaining the energy dependence of photon interactions. The DECT-derived RSP maps could potentially be compromised by image noise levels and the severity of artifacts when using physics-based mapping techniques, which would affect subsequent clinical applications. This work presents a noise-robust learning-based method to predict RSP maps from DECT for proton radiation therapy.Method:The proposed method uses a residual attention cycle-consistent generative adversarial network. Residual blocks with attention gates were used to force the model focus on the difference between RSP maps and DECT images. Cycle consistent generative adversarial networks were used to let the DECT-to-RSP mapping be close to a one-to-one mapping by introducing an inverse RSP-to-DECT mapping. To evaluate the accuracy of the method, we retrospectively investigated 20 head-and-neck cancer patients with DECT scans in proton radiation therapy simulation. Ground truth RSP values were assigned by calculation based on chemical compositions. These ground truth RSP maps acted as learning targets in the training process for DECT datasets, and were evaluated against results from the proposed method using a leave-one-out cross-validation strategy.Result:The predicted RSP maps showed an average normalized mean square error (NMSE) of 2.83% across the whole body volume, and average mean error (ME) less than 3% in all volumes of interest (VOIs). With additional simulated noise added in DECT datasets, the proposed method still maintained a comparable performance, while the physics-based stoichiometric method suffered degraded inaccuracy from increased noise level. Based on the statistical analysis of comparative dose volume histogram (DVH) metrics of dose maps calculated on ground truth and predicted RSP maps among 19 pencil beam scanning proton treatment plans, the average differences in DVH metrics for clinical target volumes (CTVs) were less than 0.2 Gy for D95% and Dmax with no statistical significance (average prescription dose = 60 Gy). Maximum difference in DVH metrics of organs-at-risk (OARs) was around 1 Gy on average. Conclusion:These results strongly indicate the high accuracy of RSP maps predicted by our machinelearning-based method and show its potential feasibility for proton treatment planning and dose calculation.

show abstract

A review on medical imaging synthesis using deep learning and its clinical applications

Wang

Lei

et al. 2020

J Applied Clin Med Phys

Self Cite

202

131

View full text Add to dashboard Cite

This paper reviewed the deep learning‐based studies for medical imaging synthesis and its clinical application. Specifically, we summarized the recent developments of deep learning‐based methods in inter‐ and intra‐modality image synthesis by listing and highlighting the proposed methods, study designs, and reported performances with related clinical applications on representative studies. The challenges among the reviewed studies were then summarized with discussion.

show abstract

MRI‐only based synthetic CT generation using dense cycle consistent generative adversarial networks

Cited by 243 publications

References 53 publications

CT prostate segmentation based on synthetic MRI‐aided deep attention fully convolution network

CT prostate segmentation based on synthetic MRI‐aided deep attention fully convolution network

Deep learning-based relative stopping power mapping generation with cone-beam CT in proton radiation therapy

A review on medical imaging synthesis using deep learning and its clinical applications

Contact Info

Product

Resources

About