Unsupervised MR-to-CT Synthesis Using Structure-Constrained CycleGAN

Yang, Haojie; Sun, Jian; Carass, Aaron; Zhao, Can; Lee, Junghoon; Prince, Jerry L.; Zhao, Xu

doi:10.1109/tmi.2020.3015379

Cited by 126 publications

(82 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These contradicting results will be discussed later. Paired models were the most adopted, with only ten studies investigating unpaired training 67,75–77,80,89,91,95,109,112 . Interestingly, Li et al.…”

Section: Resultsmentioning

confidence: 99%

“…who found such a technique beneficial against either paired or unpaired training. Yang et al 91 . found that structure‐constrained loss functions and spectral normalization ameliorated unpaired training performances in the pelvic and abdominal regions.…”

Section: Resultsmentioning

confidence: 99%

“…67,76,91 As proposed by Yang et al, 91 unsupervised training decreases the semantic information from one domain to another. 91 Such an issue may be solved by introducing a structure-consistency loss, which extracts structural features from the image defining the loss in the feature space. Yang et al's results showed improvements in this sense relative to other unsupervised methods.…”

Section: Deep-learning Considerations and Trendsmentioning

confidence: 99%

See 2 more Smart Citations

Deep learning based synthetic‐CT generation in radiotherapy and PET: A review

et al. 2021

View full text Add to dashboard Cite

Recently, deep learning (DL)-based methods for the generation of synthetic computed tomography (sCT) have received significant research attention as an alternative to classical ones. We present here a systematic review of these methods by grouping them into three categories, according to their clinical applications: I) to replace CT in magnetic resonance (MR)-based treatment planning, II) facilitate cone-beam computed tomography (CBCT)-based image-guided adaptive radiotherapy, and III) derive attenuation maps for the correction of positron emission tomography (PET).Appropriate database searching was performed on journal articles published between January 2014 and December 2020.The DL methods' key characteristics were extracted from each eligible study, and a comprehensive comparison among network architectures and metrics was reported. A detailed review of each category was given, highlighting essential contributions, identifying specific challenges, and summarising the achievements. Lastly, the statistics of all the cited works from various aspects were analysed, revealing the popularity and future trends and the potential of DL-based sCT generation. The current status of DLbased sCT generation was evaluated, assessing the clinical readiness of the presented methods.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Deep-learning Considerations and Trendsmentioning

confidence: 99%

See 1 more Smart Citation

Deep learning based synthetic‐CT generation in radiotherapy and PET: A review

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Kuang et al [ 23 ] employed an encoder to map the latent space of benign lung nodules and malignant lung nodules to guide generator to synthesize corresponding lung CT images. Yang et al [ 24 ] proposed an extra structure-consistency loss based on the modality of independent neighborhood descriptor to improve CycleGAN for unsupervised MR-to-CT synthesis. Zunair and Hamza [ 25 ] adopted adversarial training and transfer learning to convert normal and pneumonia chest X-ray to COVID-19 chest X-ray.…”

Section: Related Workmentioning

confidence: 99%

RF‐GANs: A Method to Synthesize Retinal Fundus Images Based on Generative Adversarial Network

Chen

Long

Guo

2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Diabetic retinopathy (DR) is a diabetic complication affecting the eyes, which is the main cause of blindness in young and middle-aged people. In order to speed up the diagnosis of DR, a mass of deep learning methods have been used for the detection of this disease, but they failed to attain excellent results due to unbalanced training data, i.e., the lack of DR fundus images. To address the problem of data imbalance, this paper proposes a method dubbed retinal fundus images generative adversarial networks (RF-GANs), which is based on generative adversarial network, to synthesize retinal fundus images. RF-GANs is composed of two generation models, RF-GAN1 and RF-GAN2. Firstly, RF-GAN1 is employed to translate retinal fundus images from source domain (the domain of semantic segmentation datasets) to target domain (the domain of EyePACS dataset connected to Kaggle (EyePACS)). Then, we train the semantic segmentation models with the translated images, and employ the trained models to extract the structural and lesion masks (hereafter, we refer to it as Masks) of EyePACS. Finally, we employ RF-GAN2 to synthesize retinal fundus images using the Masks and DR grading labels. This paper verifies the effectiveness of the method: RF-GAN1 can narrow down the domain gap between different datasets to improve the performance of the segmentation models. RF-GAN2 can synthesize realistic retinal fundus images. Adopting the synthesized images for data augmentation, the accuracy and quadratic weighted kappa of the state-of-the-art DR grading model on the testing set of EyePACS increase by 1.53% and 1.70%, respectively.

show abstract

“…Conventional generative adversarial networks (GANs) trained with paired image sets have been investigated most comprehensively for the generation of sCTs for RT simulations [ 20 , 21 ]. With the emergence of another deep learning technique using cycle-consistent GAN (CycGAN), which performs unpaired image-to-image translation, studies evaluating the feasibility of deep learning sCT models other than those based on GAN have been conducted [ 22 , 23 ]. However, such studies are limited to date.…”

Section: Introductionmentioning

confidence: 99%

Feasibility of Synthetic Computed Tomography Images Generated from Magnetic Resonance Imaging Scans Using Various Deep Learning Methods in the Planning of Radiation Therapy for Prostate Cancer

Yoo

Luu

Kim

et al. 2021

Cancers

View full text Add to dashboard Cite

We aimed to evaluate and compare the qualities of synthetic computed tomography (sCT) generated by various deep-learning methods in volumetric modulated arc therapy (VMAT) planning for prostate cancer. Simulation computed tomography (CT) and T2-weighted simulation magnetic resonance image from 113 patients were used in the sCT generation by three deep-learning approaches: generative adversarial network (GAN), cycle-consistent GAN (CycGAN), and reference-guided CycGAN (RgGAN), a new model which performed further adjustment of sCTs generated by CycGAN with available paired images. VMAT plans on the original simulation CT images were recalculated on the sCTs and the dosimetric differences were evaluated. For soft tissue, a significant difference in the mean Hounsfield unites (HUs) was observed between the original CT images and only sCTs from GAN (p = 0.03). The mean relative dose differences for planning target volumes or organs at risk were within 2% among the sCTs from the three deep-learning approaches. The differences in dosimetric parameters for D98% and D95% from original CT were lowest in sCT from RgGAN. In conclusion, HU conservation for soft tissue was poorest for GAN. There was the trend that sCT generated from the RgGAN showed best performance in dosimetric conservation D98% and D95% than sCTs from other methodologies.

show abstract

Unsupervised MR-to-CT Synthesis Using Structure-Constrained CycleGAN

Cited by 126 publications

References 31 publications

Deep learning based synthetic‐CT generation in radiotherapy and PET: A review

Deep learning based synthetic‐CT generation in radiotherapy and PET: A review

RF‐GANs: A Method to Synthesize Retinal Fundus Images Based on Generative Adversarial Network

Feasibility of Synthetic Computed Tomography Images Generated from Magnetic Resonance Imaging Scans Using Various Deep Learning Methods in the Planning of Radiation Therapy for Prostate Cancer

Contact Info

Product

Resources

About