Compressed Sensing: From Research to Clinical Practice With Deep Neural Networks: Shortening Scan Times for Magnetic Resonance Imaging

Abstract

Compressed sensing (CS) reconstruction methods leverage sparse structure in underlying signals to recover high-resolution images from highly undersampled measurements. When applied to magnetic resonance imaging (MRI), CS has the potential to dramatically shorten MRI scan times, increase diagnostic value, and improve overall patient experience. However, CS has several shortcomings which limit its clinical translation such as: 1) artifacts arising from inaccurate sparse modelling assumptions, 2) extensive parameter tuning required for each clinical application, and 3) clinically infeasible reconstruction times. Recently, CS has been extended to incorporate deep neural networks as a way of learning complex image priors from historical exam data. Commonly referred to as unrolled neural networks, these techniques have proven to be a compelling and practical approach to address the challenges of sparse CS. In this tutorial, we will review the classical compressed sensing formulation and outline steps needed to transform this formulation into a deep learning-based reconstruction framework. Supplementary open source code in Python will be used to demonstrate this approach with open databases. Further, we will discuss considerations in applying unrolled neural networks in the clinical setting.

Accelerating cardiac cine MRI using a deep learning‐based ESPIRiT reconstruction

“…Each ResNet is composed of 3D convolutional layers with 3 × 3 × 3 kernels to leverage all spatial and temporal dimensions for de‐aliasing. Convolutional layers are implemented using circular padding along the phase encoding and temporal directions in order to enforce circular boundary conditions in the two dimensions 24 . There are five total convolutional layers for each DL‐ESPIRiT iteration which corresponds to a spatiotemporal receptive field of size 11 × 11 × 11.…”

“…Convolutional layers are implemented using circular padding along the phase encoding and temporal directions in order to enforce circular boundary conditions in the two dimensions. 24 There are five total convolutional layers for each DL-ESPIRiT iteration which corresponds to a spatiotemporal receptive field of size 11 × 11 × 11. The first convolution of each ResNet expands the initial (2 or 4) images into 96 feature maps, which are propagated through the network until the final convolution where they are recombined into the original number of input images.…”

“…More recently, deep learning-based approaches have been proposed to leverage historical exam data from multiple subjects to implicitly and automatically learn better priors for constrained image reconstruction. [20][21][22][23][24] These methods are comprised of the following steps: (a) a conventional CS algorithm is unrolled to a fixed number of iterations, (b) the prior information in each iteration is enforced by a neural network, and (c) the unrolled algorithm is trained end-to-end in a supervised fashion. For 2D cardiac cine imaging, various neural network architectures, including cascaded 25 and recurrent unrolled networks, 26,27 have been developed and trained to reconstruct up to 9X accelerated data with higher fidelity than CS.…”

“…More recently, deep learning‐based approaches have been proposed to leverage historical exam data from multiple subjects to implicitly and automatically learn better priors for constrained image reconstruction 20‐24 . These methods are comprised of the following steps: (a) a conventional CS algorithm is unrolled to a fixed number of iterations, (b) the prior information in each iteration is enforced by a neural network, and (c) the unrolled algorithm is trained end‐to‐end in a supervised fashion.…”

“…In this work, a combined PI and deep learning‐based reconstruction framework is proposed for robust reconstruction of dynamic MRI data. The novelty of this framework is summarized as follows: (a) an extended coil sensitivity model based on ESPIRiT 28 is integrated into an existing deep learning‐based reconstruction approach 24 to improve its robustness to SENSE‐related FOV limitations and (b) a novel convolutional neural network (CNN) architecture based on separable 3D convolutions is developed to learn spatiotemporal priors for dynamic data more efficiently. We apply our novel DL‐ESPIRiT network to 12X retrospectively undersampled 2D cardiac cine data and show higher reconstruction accuracy than a state‐of‐the‐art parallel imaging and compressed sensing (PICS) algorithm with respect to standard image quality metrics.…”