Technical Note: U‐net‐generated synthetic CT images for magnetic resonance imaging‐only prostate intensity‐modulated radiation therapy treatment planning

Chen, Shupeng; Qin, An; Zhou, Dingyi; Yan, Di

doi:10.1002/mp.13247

Cited by 92 publications

(68 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chen et al applied U-net to generate sCT images for MRI-only prostate intensity-modulated radiation therapy treatment planning. 52 They reported the MAE value within body outline was 29.96 AE 4.87 HU. The MAE from our proposed method was 54.2 HU for the brain data.…”

Section: Discussionmentioning

confidence: 99%

MRI‐only based synthetic CT generation using dense cycle consistent generative adversarial networks

et al. 2019

View full text Add to dashboard Cite

Purpose Automated synthetic computed tomography (sCT) generation based on magnetic resonance imaging (MRI) images would allow for MRI‐only based treatment planning in radiation therapy, eliminating the need for CT simulation and simplifying the patient treatment workflow. In this work, the authors propose a novel method for generation of sCT based on dense cycle‐consistent generative adversarial networks (cycle GAN), a deep‐learning based model that trains two transformation mappings (MRI to CT and CT to MRI) simultaneously. Methods and materials The cycle GAN‐based model was developed to generate sCT images in a patch‐based framework. Cycle GAN was applied to this problem because it includes an inverse transformation from CT to MRI, which helps constrain the model to learn a one‐to‐one mapping. Dense block‐based networks were used to construct generator of cycle GAN. The network weights and variables were optimized via a gradient difference (GD) loss and a novel distance loss metric between sCT and original CT. Results Leave‐one‐out cross‐validation was performed to validate the proposed model. The mean absolute error (MAE), peak signal‐to‐noise ratio (PSNR), and normalized cross correlation (NCC) indexes were used to quantify the differences between the sCT and original planning CT images. For the proposed method, the mean MAE between sCT and CT were 55.7 Hounsfield units (HU) for 24 brain cancer patients and 50.8 HU for 20 prostate cancer patients. The mean PSNR and NCC were 26.6 dB and 0.963 in the brain cases, and 24.5 dB and 0.929 in the pelvis. Conclusion We developed and validated a novel learning‐based approach to generate CT images from routine MRIs based on dense cycle GAN model to effectively capture the relationship between the CT and MRIs. The proposed method can generate robust, high‐quality sCT in minutes. The proposed method offers strong potential for supporting near real‐time MRI‐only treatment planning in the brain and pelvis.

show abstract

Section: Discussionmentioning

confidence: 99%

MRI‐only based synthetic CT generation using dense cycle consistent generative adversarial networks

et al. 2019

View full text Add to dashboard Cite

show abstract

“…[13][14][15][16][17][18] Another attractive feature of deep learning techniques is that they can directly extract the relevant set of features from the data without requiring extensive feature engineering. As a result, recent papers have shown promising results from deep learning applied to sCT generation in brain [13][14][15][16][17][18] and pelvis 17,[19][20][21][22][23]…”

Section: Introductionmentioning

confidence: 99%

Patch‐based generative adversarial neural network models for head and neck MR‐only planning

et al. 2019

View full text Add to dashboard Cite

Purpose To evaluate pix2pix and CycleGAN and to assess the effects of multiple combination strategies on accuracy for patch‐based synthetic computed tomography (sCT) generation for magnetic resonance (MR)‐only treatment planning in head and neck (HN) cancer patients. Materials and methods Twenty‐three deformably registered pairs of CT and mDixon FFE MR datasets from HN cancer patients treated at our institution were retrospectively analyzed to evaluate patch‐based sCT accuracy via the pix2pix and CycleGAN models. To test effects of overlapping sCT patches on estimations, we (a) trained the models for three orthogonal views to observe the effects of spatial context, (b) we increased effective set size by using per‐epoch data augmentation, and (c) we evaluated the performance of three different approaches for combining overlapping Hounsfield unit (HU) estimations for varied patch overlap parameters. Twelve of twenty‐three cases corresponded to a curated dataset previously used for atlas‐based sCT generation and were used for training with leave‐two‐out cross‐validation. Eight cases were used for independent testing and included previously unseen image features such as fused vertebrae, a small protruding bone, and tumors large enough to deform normal body contours. We analyzed the impact of MR image preprocessing including histogram standardization and intensity clipping on sCT generation accuracy. Effects of mDixon contrast (in‐phase vs water) differences were tested with three additional cases. The sCT generation accuracy was evaluated using mean absolute error (MAE) and mean error (ME) in HU between the plan CT and sCT images. Dosimetric accuracy was evaluated for all clinically relevant structures in the independent testing set and digitally reconstructed radiographs (DRRs) were evaluated with respect to the plan CT images. Results The cross‐validated MAEs for the whole‐HN region using pix2pix and CycleGAN were 66.9 ± 7.3 vs 82.3 ± 6.4 HU, respectively. On the independent testing set with additional artifacts and previously unseen image features, whole‐HN region MAEs were 94.0 ± 10.6 and 102.9 ± 14.7 HU for pix2pix and CycleGAN, respectively. For patients with different tissue contrast (water mDixon MR images), the MAEs increased to 122.1 ± 6.3 and 132.8 ± 5.5 HU for pix2pix and CycleGAN, respectively. Our results suggest that combining overlapping sCT estimations at each voxel reduced both MAE and ME compared to single‐view non‐overlapping patch results. Absolute percent mean/max dose errors were 2% or less for the PTV and all clinically relevant structures in our independent testing set, including structures with image artifacts. Quantitative DRR comparison between planning CTs and sCTs showed agreement of bony region positions to <1 mm. Conclusions The dosimetric and MAE based accuracy, along with the similarity between DRRs from sCTs, indicate that pix2pix and CycleGAN are promising methods for MR‐only treatment planning for HN cancer. Our methods investigated for overlapping patch‐based HU estimations also ...

show abstract

“…Han employed a pre‐trained U‐Net architecture to generate sCTs for brain tumor cases. Similarly, Fu et al and Chen et al generated sCTs of the male pelvis using 2D and 3D U‐Nets. Viewing the generation of sCTs from MR images as an image‐to‐image translation problem, Maspero et al implemented a generative adversarial network (GAN) named pix2pix that has demonstrated success in this space to produce sCTs for general pelvis cases.…”

Section: Introductionmentioning

confidence: 99%

Synthetic CT reconstruction using a deep spatial pyramid convolutional framework for MR‐only breast radiotherapy

et al. 2019

View full text Add to dashboard Cite

Purpose The superior soft‐tissue contrast achieved using magnetic resonance imaging (MRI) compared to x‐ray computed tomography (CT) has led to the popularization of MRI‐guided radiation therapy (MR‐IGRT), especially in recent years with the advent of first and second generation MRI‐based therapy delivery systems for MR‐IGRT. The expanding use of these systems is driving interest in MRI‐only RT workflows in which MRI is the sole imaging modality used for treatment planning and dose calculations. To enable such a workflow, synthetic CT (sCT) data must be generated based on a patient’s MRI data so that dose calculations may be performed using the electron density information derived from CT images. In this study, we propose a novel deep spatial pyramid convolutional framework for the MRI‐to‐CT image‐to‐image translation task and compare its performance to the well established U‐Net architecture in a generative adversarial network (GAN) framework. Methods Our proposed framework utilizes atrous convolution in a method named atrous spatial pyramid pooling (ASPP) to significantly reduce the total number of parameters required to describe the model while effectively capturing rich, multi‐scale structural information in a manner that is not possible in the conventional framework. The proposed framework consists of a generative model composed of stacked encoders and decoders separated by the ASPP module, where atrous convolution is applied at increasing rates in parallel to encode large‐scale features. The performance of the proposed method is compared to that of the conventional GAN framework in terms of the time required to train the model and the image quality of the generated sCT as measured by the root mean square error (RMSE), structural similarity index (SSIM), and peak signal‐to‐noise ratio (PSNR) depending on the size of the training data set. Dose calculations based on sCT data generated using the proposed architecture are also compared to clinical plans to evaluate the dosimetric accuracy of the method. Results Significant reductions in training time and improvements in image quality are observed at every training data set size when the proposed framework is adopted instead of the conventional framework. Over 1042 test images, values of 17.7 ± 4.3 HU, 0.9995 ± 0.0003, and 71.7 ± 2.3 are observed for the RMSE, SSIM, and PSNR metrics, respectively. Dose distributions calculated based on sCT data generated using the proposed framework demonstrate passing rates equal to or greater than 98% using the 3D gamma index with a 2%/2 mm criterion. Conclusions The deep spatial pyramid convolutional framework proposed here demonstrates improved performance compared to the conventional GAN framework that has been applied to the image‐to‐image translation task of sCT generation. Adopting the method is a first step toward an MRI‐only RT workflow that enables widespread clinical applications for MR‐IGRT including online adaptive therapy.

show abstract

Technical Note: U‐net‐generated synthetic CT images for magnetic resonance imaging‐only prostate intensity‐modulated radiation therapy treatment planning

Cited by 92 publications

References 34 publications

MRI‐only based synthetic CT generation using dense cycle consistent generative adversarial networks

MRI‐only based synthetic CT generation using dense cycle consistent generative adversarial networks

Patch‐based generative adversarial neural network models for head and neck MR‐only planning

Synthetic CT reconstruction using a deep spatial pyramid convolutional framework for MR‐only breast radiotherapy

Contact Info

Product

Resources

About