Magnetic resonance imaging contrast enhancement synthesis using cascade networks with local supervision

Abstract

Purpose: Gadolinium-based contrast agents (GBCAs) are widely administrated in MR imaging for diagnostic studies and treatment planning. Although GBCAs are generally thought to be safe, various health and environmental concerns have been raised recently about their use in MR imaging. The purpose of this work is to derive synthetic contrast enhance MR images from unenhanced counterpart images, thereby eliminating the need for GBCAs, using a cascade deep learning workflow that incorporates contour information into the network. Methods and materials: The proposed workflow consists of two sequential networks: (1) a retina U-Net, which is first trained to derive semantic features from the non-contrast MR images in representing the tumor regions; and (2) a synthesis module, which is trained after the retina U-Net to take the concatenation of the semantic feature maps and non-contrast MR image as input and to generate the synthetic contrast enhanced MR images. After network training, only the non-contrast enhanced MR images are required for the input in the proposed workflow. The MR images of 369 patients from the multimodal brain tumor segmentation challenge 2020 (BraTS2020) dataset were used in this study to evaluate the proposed workflow for synthesizing contrast enhanced MR images (200 patients for five-fold cross-validation and 169 patients for holdout test). Quantitative evaluations were conducted by calculating the normalized mean absolute error (NMAE), structural similarity index measurement (SSIM), and Pearson correlation coefficient (PCC). The original contrast enhanced MR images were considered as the ground truth in this analysis. Results: The proposed cascade deep learning workflow synthesized contrast enhanced MR images that are not visually differentiable from the ground truth with and without supervision of the tumor contours during the network training. Difference images and profiles of the synthetic contrast enhanced MR images revealed that intensity differences could be observed in the tumor region if the contour information was not incorporated in network training. Among the holdout test patients, mean values and standard deviations of the NMAE, SSIM, and PCC were 0.063±0.022, 0.991±0.007 and 0.995±0.006, respectively, for the whole brain; and were 0.050±0.025, 0.993±0.008 and 0.999±0.003, respectively, for the tumor contour regions. Quantitative evaluations with five-fold crossvalidation and hold-out test showed that the calculated metrics can be significantly enhanced (p-values ≤ 0.002) with the tumor contour supervision in network training. Conclusion:The proposed workflow was able to generate synthetic contrast enhanced MR images that closely resemble the ground truth images from non-contrast enhanced MR images when the network training included tumor 3278

Artificial Contrast

“…This can only be assessed by reading studies. The publications on zero-dose approaches in particular are mainly methodological papers that do not include a structured qualitative analysis by radiologists 48–51 . The criteria used in qualitative analysis may differ significantly between studies and are therefore often difficult to compare.…”

“…To the best of our knowledge, all methods essentially follow the preprocessing pipeline proposed by Kleesiek et al 57 In terms of network architecture, most publications advocate a 3D U-Net model. Chen et al 71 applied a high-resolution fully convolutional network motivated by HR-nets to extract deep high-resolution features, whereas Xie et al use a cascade of a retina U-Net and a U-Net 48,49 . In contrast to these convolutional models, Liu et al 38 recently proposed a conditional Swin transformer to synthesize 2D MRI images including full-dose T1w images 51 .…”

“…In detail, a point-wise binary mask extracted voxels where the difference between the full-dose T1w image and the unenhanced T1w image is larger than a certain threshold. Alternatively, Xie et al 48 emphasized the synthesis on pathological contrast areas using regions of interest extracted from the BraTS dataset labels.…”

Toward deep learning replacement of gadolinium in neuro-oncology: A review of contrast-enhanced synthetic MRI

“…In contrast, the implicit motion-artifact correction and aliasing suppression are upsides of these methods. Other methods carry out the synthesis from only a few native image modalities (Dai et al, 2020;Sharma and Hamarneh, 2020;Chen et al, 2022) or even from only one native input (Xie et al, 2022). However, these methods were solely validated with image quality metrics (SSIM, PSNR, etc.…”

Synthetic Inflammation Imaging with PatchGAN Deep Learning Networks

“…Another approach in eliminating Gd dosage for brain MRI used an innovative training scheme, training a network for tumor detection and passing convolutional feature maps from that network as inputs to a conventional image synthesis architecture. This allowed the image synthesis architecture to focus on pathologic regions when optimizing parameters to produce synthetic post-contrast images [26]. Some approaches beyond image synthesis have also been investigated to eliminate the need for Gd administration.…”