3D High-Quality Magnetic Resonance Image Restoration in Clinics Using Deep Learning

Abstract

Shortening acquisition time and reducing the motion-artifact are two of the most essential concerns in magnetic resonance imaging. As a promising solution, deep learning-based high-quality MR image restoration has been investigated to generate higher resolution MR images from lower resolution images acquired with shortened acquisition time and free of motion-artifact, without costing additional acquisition time, modifying the pulse sequences or repeating the acquisition. However, there are still numerous problems preventing deep learning approaches from becoming practical in the clinic environment. Specifically, most of the prior works focus solely on the network model but ignore the impact of various downsampling strategies on the acquisition time. Besides, the long inference time and high GPU consumption are also the bottle neck to deploy most of the prior works in clinics. Furthermore, prior studies employ random movement in retrospective motion artifact generation, resulting in uncontrollable severity of motion artifact. More importantly, doctors are unsure whether the generated MR images are trustworthy, making diagnosis difficult. To overcome all the aforementioned problems, we employed a unified deep learning neural network with 2D convolutional filter for both 3D MRI super resolution and motion artifact reduction, demonstrating such a unified framework can achieve better performance in 3D MRI restoration task compared to other states of the art methods and remains the GPU consumption and inference time significantly low, thus easier to be deployed.We also analyzed several downsampling strategies based on acceleration factor, including multiple combinations of in-plane and through-plane downsampling, and developed a controllable and quantifiable motion artifact generation method. At last, the pixel-wise uncertainty was calculated and used to estimate the accuracy of generated images, providing additional information for a reliable diagnosis.

Unsupervised Representation Learning for 3D MRI Super Resolution with Degradation Adaptation

“…To demonstrate the negative impact from unavailability of paired MRI HR and LR images and degradation shift, and effectiveness of our proposed method, three supervised training approaches were employed as the benchmarks using TS-RCAN [8] in the supervised manner to reconstruct SR MRI images:…”

“…For the encoders, we used 6 convolution layers with Leaky ReLU between each two layers, and the head layer of the downsampling feature extractor downsamples the input HR image to the same size of the LR image. We used the TS-RCAN [8] with 5 residual groups (RG) and 5 residual blocks (RCAB) in each RG as the backbone of the decoders, and VGG [23] as the discriminators for UDEAN. TS-RCAN is modified from RCAN [21] [22] to conduct 3D MRI SR task with very low consumption of computation resource and short inference time.…”

“…Most of previous works in MRI SR reconstruction follows this routine with designing different network models: Both DCSRN [2] and CSN [4] utilize DenseNet [25] to enhance the feature fusion for the reconstruction procedure; mDCSRN [3], as the extension of DCSRN, increases the depth of SR reconstruction network and applies discriminator to facilitate its capability; the authors of [5] incorporates the spatial gradient of LR images as prior to guide the MRI SR reconstruction; SERAN [6] employs attention mechanism into MRI SR reconstruction procedure; and leveraging of HR reference from another modality as prior information is also introduced on top of attention based network in MINet [7] to achieve outstanding performance. Besides, TS-RCAN [8] achieves the slightly better performance compare to MINet with only single modality data and great reduction of demands on computational resource and inference time.…”

UNAEN: Unsupervised Abnomality Extraction Network for MRI Motion Artifact Reduction

“…We referred to the paper [37] to simulate the motion in MR images. The method of splicing lines from multiple K-space was used to simulate the generation of real motion artifacts.…”

The Unreasonable Effectiveness of Deep Evidential Regression

“…A different approach to uncertainty-aware NNs may be useful to more efficiency quantify, and also to disentangle, the several types of uncertainties: Deep Evidential Regression (DER) aims to simultaneously predict both uncertainty types in a single forward pass without sampling or utilization of out-of-distribution data, based on learning evidential distributions for aleatoric and epistemic uncertainties (Amini et al 2020). Yet only with simple empirical demonstrations on univariate regression tasks, this technique has already been applied and recommended in medical and other safety critical applications (Liu et al 2021;Soleimany et al 2021;Cai et al 2021;Chen, Bromuri, and van Eekelen 2021;Singh et al 2022;Petek et al 2022;Li and Liu 2021). With an alternative derivation and experimentation, we identify theoretical shortcomings that do not justify the empirical results let alone the assumed reliability in practice -it can be vital to understand to what degree the uncertainty estimations are trustworthy.…”