One Model to Synthesize Them All: Multi-Contrast Multi-Scale Transformer for Missing Data Imputation

Liu, Jiang; Pasumarthi, Srivathsa; Duffy, Ben; Gong, Enhao; Datta, Keshav; Zaharchuk, Greg

doi:10.1109/tmi.2023.3261707

Cited by 32 publications

(11 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, the addition of the ADC map by Jayachandran Preetha et al 50 did not improve the SSIM. As in the work of Liu et al, 51 Jayachandran Preetha et al 50 reported that the unenhanced T1w, the FLAIR, and the T2w sequences were most important, in that order.…”

Section: Resultsmentioning

confidence: 68%

“…The publications on zero-dose approaches in particular are mainly methodological papers that do not include a structured qualitative analysis by radiologists. [48][49][50][51] The criteria used in qualitative analysis may differ significantly between studies and are therefore often difficult to compare. Before moving to the approaches to reduce the use of GBCA, we would like to introduce and briefly discuss the common methods for quantitative and qualitative evaluation of the artificial contrast-enhanced T1w sequences.…”

Section: Methods Of Evaluating Artificial Imagesmentioning

confidence: 99%

“…the third column of Table 2. The only method that just used unenhanced T1w images is Xie et al 48 Chen et al 49 additionally used T2w images and ADC maps, as suggested by Kleesiek et al 57 Liu et al 51 predicted artificial full-dose images using T1w, T2w, and FLAIR images. The combination of T1w, T2w, ADC, and FLAIR images as inputs was investigated by Jayachandran Preetha et al, 50 whereas Calabrese et al 69 additionally considered arterial spin labeling, mean diffusivity, and fractional anisotropy images.…”

Section: Methodsmentioning

confidence: 99%

“…48,49 In contrast to these convolutional models, Liu et al 38 recently proposed a conditional Swin transformer to synthesize 2D MRI images including full-dose T1w images. 51 This approach has the advantage that only 1 model is sufficient to synthesize all other sequences independent of which one is missing.…”

Section: Methodsmentioning

confidence: 99%

“…As previously mentioned, these methods were also used to synthesize postcontrast T1w images. 51 Synthesis models based on previous inputs such as unenhanced T1w, T2w, or FLAIR images are denoted as conditional GANs. For a comprehensive technical review of GANs, we refer the interested reader to Gui et al, 77 and for a medical focus, to Kazeminia et al 78 Similarly to the low-dose approach, numerous different loss terms and combinations thereof were proposed to focus neural networks on predicting relevant contrast agent signals.…”

Section: Methodsmentioning

confidence: 99%

See 4 more Smart Citations

Artificial Contrast

et al. 2023

View full text Add to dashboard Cite

Deep learning approaches are playing an ever-increasing role throughout diagnostic medicine, especially in neuroradiology, to solve a wide range of problems such as segmentation, synthesis of missing sequences, and image quality improvement. Of particular interest is their application in the reduction of gadolinium-based contrast agents, the administration of which has been under cautious reevaluation in recent years because of concerns about gadolinium deposition and its unclear long-term consequences. A growing number of studies are investigating the reduction (low-dose approach) or even complete substitution (zero-dose approach) of gadolinium-based contrast agents in diverse patient populations using a variety of deep learning methods. This work aims to highlight selected research and discusses the advantages and limitations of recent deep learning approaches, the challenges of assessing its output, and the progress toward clinical applicability distinguishing between the low-dose and zero-dose approach.

show abstract

Section: Resultsmentioning

confidence: 68%

Section: Methods Of Evaluating Artificial Imagesmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

See 3 more Smart Citations

Artificial Contrast

et al. 2023

View full text Add to dashboard Cite

show abstract

Reducing Gadolinium Contrast With Artificial Intelligence

Tsui,

Calabrese,

Zaharchuk

et al. 2023

Magnetic Resonance Imaging

Self Cite

View full text Add to dashboard Cite

Gadolinium contrast is an important agent in magnetic resonance imaging (MRI), particularly in neuroimaging where it can help identify blood–brain barrier breakdown from an inflammatory, infectious, or neoplastic process. However, gadolinium contrast has several drawbacks, including nephrogenic systemic fibrosis, gadolinium deposition in the brain and bones, and allergic‐like reactions. As computer hardware and technology continues to evolve, machine learning has become a possible solution for eliminating or reducing the dose of gadolinium contrast. This review summarizes the clinical uses of gadolinium contrast, the risks of gadolinium contrast, and state‐of‐the‐art machine learning methods that have been applied to reduce or eliminate gadolinium contrast administration, as well as their current limitations, with a focus on neuroimaging applications.Evidence Level3Technical EfficacyStage 1

show abstract

T1‐contrast enhanced MRI generation from multi‐parametric MRI for glioma patients with latent tumor conditioning

Eidex,

Safari,

Qiu

et al. 2024

Medical Physics

View full text Add to dashboard Cite

BackgroundGadolinium‐based contrast agents (GBCAs) are commonly used in MRI scans of patients with gliomas to enhance brain tumor characterization using T1‐weighted (T1W) MRI. However, there is growing concern about GBCA toxicity. This study develops a deep‐learning framework to generate T1‐postcontrast (T1C) from pre‐contrast multiparametric MRI.PurposeWe propose the tumor‐aware vision transformer (TA‐ViT) model that predicts high‐quality T1C images. The predicted tumor region is significantly improved (p < 0.001) by conditioning the transformer layers from predicted segmentation maps through the adaptive layer norm zero mechanism. The predicted segmentation maps were generated with the multi‐parametric residual (MPR) ViT model and transformed into a latent space to produce compressed, feature‐rich representations. The TA‐ViT model was applied to T1w and T2‐FLAIR to predict T1C MRI images of 501 glioma cases from an open‐source dataset. Selected patients were split into training (N = 400), validation (N = 50), and test (N = 51) sets. Model performance was evaluated with the peak‐signal‐to‐noise ratio (PSNR), normalized cross‐correlation (NCC), and normalized mean squared error (NMSE).ResultsBoth qualitative and quantitative results demonstrate that the TA‐ViT model performs superior against the benchmark MPR‐ViT model. Our method produces synthetic T1C MRI with high soft tissue contrast and more accurately synthesizes both the tumor and whole brain volumes. The synthesized T1C images achieved remarkable improvements in both tumor and healthy tissue regions compared to the MPR‐ViT model. For healthy tissue and tumor regions, the results were as follows: NMSE: 8.53 ± 4.61E‐4; PSNR: 31.2 ± 2.2; NCC: 0.908 ± 0.041 and NMSE: 1.22 ± 1.27E‐4, PSNR: 41.3 ± 4.7, and NCC: 0.879 ± 0.042, respectively.ConclusionThe proposed method generates synthetic T1C images that closely resemble real T1C images. Future development and application of this approach may enable contrast‐agent‐free MRI for brain tumor patients, eliminating the risk of GBCA toxicity and simplifying the MRI scan protocol.

show abstract

One Model to Synthesize Them All: Multi-Contrast Multi-Scale Transformer for Missing Data Imputation

Cited by 32 publications

References 49 publications

Artificial Contrast

Artificial Contrast

Reducing Gadolinium Contrast With Artificial Intelligence

T1‐contrast enhanced MRI generation from multi‐parametric MRI for glioma patients with latent tumor conditioning

Contact Info

Product

Resources

About