Synthesizing MR Image Contrast Enhancement Using 3D High-Resolution ConvNets

Chen, Chao; Raymond, Catalina; Speier, Bill; Jin, Xinyu; Cloughesy, Timothy F.; Enzmann, Dieter R.; Ellingson, Benjamin M.; Arnold, Corey

doi:10.1109/tbme.2022.3192309

Cited by 35 publications

(58 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the contour information incorporated into the network training, all of the investigated numerical metrics were improved in the sT1CE pro images compared with the sT1CE woc images for both the whole brain and tumor regions. In addition, the proposed method outperformed the frameworks proposed by Chen et al 37 . (sT1CE Chen ) and Wang et al 39 .…”

Section: Resultsmentioning

confidence: 67%

Magnetic resonance imaging contrast enhancement synthesis using cascade networks with local supervision

et al. 2022

View full text Add to dashboard Cite

Purpose: Gadolinium-based contrast agents (GBCAs) are widely administrated in MR imaging for diagnostic studies and treatment planning. Although GBCAs are generally thought to be safe, various health and environmental concerns have been raised recently about their use in MR imaging. The purpose of this work is to derive synthetic contrast enhance MR images from unenhanced counterpart images, thereby eliminating the need for GBCAs, using a cascade deep learning workflow that incorporates contour information into the network. Methods and materials: The proposed workflow consists of two sequential networks: (1) a retina U-Net, which is first trained to derive semantic features from the non-contrast MR images in representing the tumor regions; and (2) a synthesis module, which is trained after the retina U-Net to take the concatenation of the semantic feature maps and non-contrast MR image as input and to generate the synthetic contrast enhanced MR images. After network training, only the non-contrast enhanced MR images are required for the input in the proposed workflow. The MR images of 369 patients from the multimodal brain tumor segmentation challenge 2020 (BraTS2020) dataset were used in this study to evaluate the proposed workflow for synthesizing contrast enhanced MR images (200 patients for five-fold cross-validation and 169 patients for holdout test). Quantitative evaluations were conducted by calculating the normalized mean absolute error (NMAE), structural similarity index measurement (SSIM), and Pearson correlation coefficient (PCC). The original contrast enhanced MR images were considered as the ground truth in this analysis. Results: The proposed cascade deep learning workflow synthesized contrast enhanced MR images that are not visually differentiable from the ground truth with and without supervision of the tumor contours during the network training. Difference images and profiles of the synthetic contrast enhanced MR images revealed that intensity differences could be observed in the tumor region if the contour information was not incorporated in network training. Among the holdout test patients, mean values and standard deviations of the NMAE, SSIM, and PCC were 0.063±0.022, 0.991±0.007 and 0.995±0.006, respectively, for the whole brain; and were 0.050±0.025, 0.993±0.008 and 0.999±0.003, respectively, for the tumor contour regions. Quantitative evaluations with five-fold crossvalidation and hold-out test showed that the calculated metrics can be significantly enhanced (p-values ≤ 0.002) with the tumor contour supervision in network training. Conclusion:The proposed workflow was able to generate synthetic contrast enhanced MR images that closely resemble the ground truth images from non-contrast enhanced MR images when the network training included tumor 3278

show abstract

Section: Resultsmentioning

confidence: 67%

Magnetic resonance imaging contrast enhancement synthesis using cascade networks with local supervision

et al. 2022

View full text Add to dashboard Cite

show abstract

“…• Previous studies 22,25,26 utilized multiple MR sequences as input, which is costly to patients in terms of time and money. Meanwhile, different MR modalities may suffer from registration errors.…”

Section: Discussionmentioning

confidence: 99%

“…In comparison with existing CE synthesis approaches, there are several decisive advantages to our model: Previous studies 22,25,26 utilized multiple MR sequences as input, which is costly to patients in terms of time and money. Meanwhile, different MR modalities may suffer from registration errors.…”

Section: Discussionmentioning

confidence: 99%

“…Our CE synthesis method is compared with 3D Res U‐Net 25 Bayes U‐Net 22 and Convnets, 26 which are commonly used and performed well in the field. All the baseline models were evaluated on datasets A and B.…”

Section: Methodsmentioning

confidence: 99%

“…Samy et al 25 trained a 3D U-Net to predict full-dose CE-T1w images from zero dose and 25% dose MRI sequences, and they further evaluated the performance of predicted images in terms of lesion detection performance. Chen et al 26 developed a 3D high-resolution (HR) fully convolutional network (FCN) to predict CE-T1w scans from three noncontrast MR sequences, and they also improved the pixel-wise loss by weighting the tumor regions to re-balance the contribution of these small but essential regions. Wang et al 27 integrated an attention network into a cycle-consistent adversarial network (CycleGAN) to learn the mapping between noncontrast MR images and their post-contrast counterparts.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep learning‐based 3D MRI contrast‐enhanced synthesis from a 2D noncontrast T2Flair sequence

Wang

Yang

et al. 2022

Medical Physics

View full text Add to dashboard Cite

Purpose Gadolinium‐based contrast agents (GBCAs) have been successfully applied in magnetic resonance (MR) imaging to facilitate better lesion visualization. However, gadolinium deposition in the human brain raised widespread concerns recently. On the other hand, although high‐resolution three‐dimensional (3D) MR images are more desired for most existing medical image processing algorithms, their long scan duration and high acquiring costs make 2D MR images still much more common clinically. Therefore, developing alternative solutions for 3D contrast‐enhanced MR image synthesis to replace GBCAs injection becomes an urgent requirement. Methods This study proposed a deep learning framework that produces 3D isotropic full‐contrast T2Flair images from 2D anisotropic noncontrast T2Flair image stacks. The super‐resolution (SR) and contrast‐enhanced (CE) synthesis tasks are completed in sequence by using an identical generative adversarial network (GAN) with the same techniques. To solve the problem that intramodality datasets from different scanners have specific combinations of orientations, contrasts, and resolutions, we conducted a region‐based data augmentation technique on the fly during training to simulate various imaging protocols in the clinic. We further improved our network by introducing atrous spatial pyramid pooling, enhanced residual blocks, and deep supervision for better quantitative and qualitative results. Results Our proposed method achieved superior CE‐synthesized performance in quantitative metrics and perceptual evaluation. In detail, the PSNR, structural‐similarity‐index, and AUC are 32.25 dB, 0.932, and 0.991 in the whole brain and 24.93 dB, 0.851, and 0.929 in tumor regions. The radiologists’ evaluations confirmed that our proposed method has high confidence in the diagnosis. Analysis of the generalization ability showed that benefiting from the proposed data augmentation technique, our network can be applied to “unseen” datasets with slight drops in quantitative and qualitative results. Conclusion Our work demonstrates the clinical potential of synthesizing diagnostic 3D isotropic CE brain MR images from a single 2D anisotropic noncontrast sequence.

show abstract

Contrast‐enhanced MRI synthesis using dense‐dilated residual convolutions based 3D network toward elimination of gadolinium in neuro‐oncology

Osman

Tamam

2023

J Applied Clin Med Phys

View full text Add to dashboard Cite

Recent studies have raised broad safety and health concerns about using of gadolinium contrast agents during magnetic resonance imaging (MRI) to enhance identification of active tumors. In this paper, we developed a deep learning‐based method for three‐dimensional (3D) contrast‐enhanced T1‐weighted (T1) image synthesis from contrast‐free image(s). The MR images of 1251 patients with glioma from the RSNA‐ASNR‐MICCAI BraTS Challenge 2021 dataset were used in this study. A 3D dense‐dilated residual U‐Net (DD‐Res U‐Net) was developed for contrast‐enhanced T1 image synthesis from contrast‐free image(s). The model was trained on a randomly split training set (n = 800) using a customized loss function and validated on a validation set (n = 200) to improve its generalizability. The generated images were quantitatively assessed against the ground‐truth on a test set (n = 251) using the mean absolute error (MAE), mean‐squared error (MSE), peak signal‐to‐noise ratio (PSNR), structural similarity (SSIM), normalized mutual information (NMI), and Hausdorff distance (HDD) metrics. We also performed a qualitative visual similarity assessment between the synthetic and ground‐truth images. The effectiveness of the proposed model was compared with a 3D U‐Net baseline model and existing deep learning‐based methods in the literature. Our proposed DD‐Res U‐Net model achieved promising performance for contrast‐enhanced T1 synthesis in both quantitative metrics and perceptual evaluation on the test set (n = 251). Analysis of results on the whole brain region showed a PSNR (in dB) of 29.882 ± 5.924, a SSIM of 0.901 ± 0.071, a MAE of 0.018 ± 0.013, a MSE of 0.002 ± 0.002, a HDD of 2.329 ± 9.623, and a NMI of 1.352 ± 0.091 when using only T1 as input; and a PSNR (in dB) of 30.284 ± 4.934, a SSIM of 0.915 ± 0.063, a MAE of 0.017 ± 0.013, a MSE of 0.001 ± 0.002, a HDD of 1.323 ± 3.551, and a NMI of 1.364 ± 0.089 when combining T1 with other MRI sequences. Compared to the U‐Net baseline model, our model revealed superior performance. Our model demonstrated excellent capability in generating synthetic contrast‐enhanced T1 images from contrast‐free MR image(s) of the whole brain region when using multiple contrast‐free images as input. Without incorporating tumor mask information during network training, its performance was inferior in the tumor regions compared to the whole brain which requires further improvements to replace the gadolinium administration in neuro‐oncology.

show abstract

Synthesizing MR Image Contrast Enhancement Using 3D High-Resolution ConvNets

Cited by 35 publications

References 47 publications

Magnetic resonance imaging contrast enhancement synthesis using cascade networks with local supervision

Magnetic resonance imaging contrast enhancement synthesis using cascade networks with local supervision

Deep learning‐based 3D MRI contrast‐enhanced synthesis from a 2D noncontrast T2Flair sequence

Contrast‐enhanced MRI synthesis using dense‐dilated residual convolutions based 3D network toward elimination of gadolinium in neuro‐oncology

Contact Info

Product

Resources

About