Physics-driven deep training of dictionary-based algorithms for MR image reconstruction

Ravishankar, Saiprasad; Chun, Il Yong; Fessier, Jeffrey A.

doi:10.1109/acssc.2017.8335685

Cited by 17 publications

(43 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, learned CNNs (deep learning) may not typically or rigorously incorporate the imaging measurement model or the information about the Physics of the imaging process, which are a key part of solving inverse problems. Hence, there has been recent interest in learning the parameters of iterative algorithms that solve regularized inverse problems [35], [142] (cf. Section VI for more such methods).…”

Section: F Physics-driven Deep Training Of Transform-based Reconstrumentioning

confidence: 99%

“…Recent works have interpreted early transform-based BCS algorithms as deep physics-driven convolutional networks learned on-the-fly, i.e., in a blind manner, from measurements [35], [148]. For example, the image update step in the square transform BCS (that learns a unitary transform) algorithm in [29] involves a least squares-type optimization with the following normal equation:…”

Section: F Physics-driven Deep Training Of Transform-based Reconstrumentioning

confidence: 99%

“…Once learned, the reconstruction model can be efficiently applied to test data using convo-lutions, thresholding, and least squares-type updates. While [35], [148] did not enforce the corresponding synthesis and transform filters (in each arm of the decorruption module) to be matched in each layer, recent work [152] learned matched filters, improving image quality. The learning of such physicsdriven networks is an active area of research, with interesting possibilities for new innovation in the convolutional models in the architecture motivated by more recent transform and dictionary learning based (or other) reconstruction methods.…”

Section: F Physics-driven Deep Training Of Transform-based Reconstrumentioning

confidence: 99%

“…A high level illustration of input space partitioning for the three layer neural network with two filter channels for each layer. Input images at each partition share the same linear representation, but not across different partitions.combinatorial nature of the expansion in(35) due to the ReLU.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Image Reconstruction: From Sparsity to Data-Adaptive Methods and Machine Learning

2020

Self Cite

View full text Add to dashboard Cite

The field of medical image reconstruction has seen roughly four types of methods. The first type tended to be analytical methods, such as filtered back-projection (FBP) for X-ray computed tomography (CT) and the inverse Fourier transform for magnetic resonance imaging (MRI), based on simple mathematical models for the imaging systems. These methods are typically fast, but have suboptimal properties such as poor resolution-noise trade-off for CT. A second type is iterative reconstruction methods based on more complete models for the imaging system physics and, where appropriate, models for the sensor statistics. These iterative methods improved image quality by reducing noise and artifacts. The FDA-approved methods among these have been based on relatively simple regularization models. A third type of methods has been designed to accommodate modified data acquisition methods, such as reduced sampling in MRI and CT to reduce scan time or radiation dose. These methods typically involve mathematical image models involving assumptions such as sparsity or lowrank. A fourth type of methods replaces mathematically designed models of signals and systems with data-driven or adaptive models inspired by the field of machine learning. This paper focuses on the two most recent trends in medical image reconstruction: methods based on sparsity or low-rank models, and data-driven methods based on machine learning techniques.

show abstract

Section: F Physics-driven Deep Training Of Transform-based Reconstrumentioning

confidence: 99%

Section: F Physics-driven Deep Training Of Transform-based Reconstrumentioning

confidence: 99%

Section: F Physics-driven Deep Training Of Transform-based Reconstrumentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Image Reconstruction: From Sparsity to Data-Adaptive Methods and Machine Learning

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…The most recent trend involves supervised (e.g., deep) learning of MRI models such as those based on convolutional neural networks [10,[29][30][31][32]. Some of these works incorporate the measurement forward model (physics) in the reconstruction model that is typically an unrolled iterative algorithm [8][9][10]. Supervised learning of TL-MRI models has also shown promise [9,33].…”

Section: B Data-driven or Learning-based Models For Reconstructionmentioning

confidence: 99%

Transform Learning for Magnetic Resonance Image Reconstruction: From Model-Based Learning to Building Neural Networks

Wen

Ravishankar

Pfister

et al. 2020

IEEE Signal Process. Mag.

Self Cite

View full text Add to dashboard Cite

Magnetic resonance imaging (MRI) is widely used in clinical practice, but it has been traditionally limited by its slow data acquisition. Recent advances in compressed sensing (CS) techniques for MRI reduce acquisition time while maintaining high image quality. Whereas classical CS assumes the images are sparse in known analytical dictionaries or transform domains, methods using learned image models for reconstruction have become popular. The model could be pre-learned from datasets, or learned simultaneously with the reconstruction, i.e., blind CS (BCS). Besides the well-known synthesis dictionary model, recent advances in transform learning (TL) provide an efficient alternative framework for sparse modeling in MRI. TL-based methods enjoy numerous advantages including exact sparse coding, transform update, and clustering solutions, cheap computation, and convergence guarantees, and provide high-quality results in MRI compared to popular competing methods. This paper provides a review of some recent works in MRI reconstruction from limited data, with focus on the recent TL-based methods. A unified framework for incorporating various TL-based models is presented. We discuss the LASSI Ravishankar et al. PANO Qu et al. 2013 PBDW Qu et al. Level-set-based CS Ye et al. 2002 Semi-Adaptive CS MRI PSF Imaging Liang LORAKS Haldar V-Net Hammernik et al. LOSTAkc akaya et al.

show abstract

Deep supervised dictionary learning by algorithm unrolling—Application to fast 2D dynamic MR image reconstruction

et al. 2023

View full text Add to dashboard Cite

Background Unrolled neural networks (NNs) have been extensively applied to different image reconstruction problems across all imaging modalities. A key component of the latter is that they allow for physics‐informed learning of the regularization method, which is parametrized by the NN. However, due to the lack of understanding of deep NNs from a theoretical point of view, unrolled NNs are still black‐boxes when the regularizers are given by deep NNs, for example, U‐Nets. Purpose Dictionarylearning (DL) is a well‐established regularization method, which is based on learning a transform to sparsely approximate the signals of interest. Typically, DL‐based image reconstruction either employs a dictionary, which was pretrained on a set of patches which were extracted from ground‐truth images or a dictionary which is jointly trained during the reconstruction. However, in both cases, the used DL‐algorithms are not designed to take into account the reconstruction problem or the underlying physical model, which describes the imaging process. In this work, we propose a DL‐algorithm based on unrolled NNs to overcome these limitations. Methods We construct an unrolled NN, which corresponds to an unrolled DL‐based reconstruction algorithm and train the unrolled NN to optimize its weights, that is, the atoms of the dictionary, by back‐propagation in a supervised manner. Further, we propose a new way to employ a 2D dictionary in the spatio‐temporal domain. We tested and evaluated the method on an accelerated cardiac cine MR image reconstruction problem using 216/36/36 dynamic images for training, validation, and testing and compared it to two well‐known state‐of‐the‐art approaches for cardiac cine MRI based on deep iterative CNNs. Further, we analyze the obtained dictionaries in terms of dictionary‐coherence and structure of the atoms. Last, we compare the reported methods in terms of stability by applying them to an entirely different dataset consisting of 49 different test images. Results The investigated physics‐informed DL‐approach yields significantly more accurate reconstructions compared to the DL‐method, which uses dictionaries obtained by decoupled pretraining, thereby providing an improvement of up to 4.90 dB in terms of PSNR and 5% in terms of SSIM. Further, the proposed spatio‐temporal 2D dictionary outperforms the 1D and 3D dictionaries by preventing smoothing of image details while still accurately removing undersampling artifacts and noise resulting in an increase of up to 1.10 dB in terms of PSNR and 4% in terms of SSIM. Although being surpassed by the CNNs on the first dataset, the proposed NNs‐based DL method is more stable compared to the latter approach and yields comparable results on the second dataset. Last, it has the advantage of being entirely interpretable in each component. Conclusions The presented physics‐informed NN can be used as training algorithm for a classical and interpretable data‐driven regularization method based on a learned dictionary, which can then not only be linked to the ...

show abstract

Physics-driven deep training of dictionary-based algorithms for MR image reconstruction

Cited by 17 publications

References 12 publications

Image Reconstruction: From Sparsity to Data-Adaptive Methods and Machine Learning

Image Reconstruction: From Sparsity to Data-Adaptive Methods and Machine Learning

Transform Learning for Magnetic Resonance Image Reconstruction: From Model-Based Learning to Building Neural Networks

Deep supervised dictionary learning by algorithm unrolling—Application to fast 2D dynamic MR image reconstruction

Contact Info

Product

Resources

About