Efficient Sum of Outer Products Dictionary Learning (SOUP-DIL) and Its Application to Inverse Problems

Abstract

The sparsity of signals in a transform domain or dictionary has been exploited in applications such as compression, denoising and inverse problems. More recently, data-driven adaptation of synthesis dictionaries has shown promise compared to analytical dictionary models. However, dictionary learning problems are typically non-convex and NP-hard, and the usual alternating minimization approaches for these problems are often computationally expensive, with the computations dominated by the NP-hard synthesis sparse coding step. This paper exploits the ideas that drive algorithms such as K-SVD, and investigates in detail efficient methods for aggregate sparsity penalized dictionary learning by first approximating the data with a sum of sparse rank-one matrices (outer products) and then using a block coordinate descent approach to estimate the unknowns. The resulting block coordinate descent algorithms involve efficient closed-form solutions. Furthermore, we consider the problem of dictionary-blind image reconstruction, and propose novel and efficient algorithms for adaptive image reconstruction using block coordinate descent and sum of outer products methodologies. We provide a convergence study of the algorithms for dictionary learning and dictionary-blind image reconstruction. Our numerical experiments show the promising performance and speedups provided by the proposed methods over previous schemes in sparse data representation and compressed sensing-based image reconstruction.

Image Reconstruction: From Sparsity to Data-Adaptive Methods and Machine Learning

“…Tensor-structured (patch-based) dictionary learning has also been exploited recently for dynamic CT [120] and spectral CT [121] reconstructions. 4) Recent Efficient Dictionary Learning-Based Methods: Recent work proposed efficient dictionary learning-based reconstruction algorithms, dubbed SOUP-DIL image reconstruction algorithms [122] that used the following regularizer:…”

“…While the earlier DL-MRI used inexact (greedy) and expensive sparse code updates and lacked convergence analysis, the SOUP-DIL scheme used efficient, exact updates and was proved to converge to the critical points (generalized stationary points) of the underlying problems and improved image quality over several schemes [122]. Fig.…”

Transform Learning for Magnetic Resonance Image Reconstruction: From Model-Based Learning to Building Neural Networks

“…Recent works have also developed efficient synthesis dictionary learning-based reconstruction schemes such as SOUP-DIL MRI [28], and LASSI [22] that uses a low-rank + learned dictionary-sparse model for dynamic MRI. The most recent trend involves supervised (e.g., deep) learning of MRI models such as those based on convolutional neural networks [10,[29][30][31][32].…”

“…The patch size used in STL-MRI was 8 × 8 for a fair comparison to STROLLR-MRI. The DL-MRI settings are as used in[28]. DL-MRI, STL-MRI, and STROLLR-MRI follow the BCS settings.…”

Online Adaptive Image Reconstruction (OnAIR) Using Dictionary Models

“…where P j ∈ C n×M is the matrix whose columns contain the patches P l x j for 1 ≤ l ≤ M , and c t i is the ith column of C t . We use a block coordinate descent approach [47] (with few iterations) to update the sparse coefficients c t i and atoms d i (columns of D) in (3) sequentially. For each 1 ≤ i ≤ m, we first minimize (3) with respect to c t i keeping the other variables fixed (sparse coding step), and then we update d i keeping the other variables fixed (dictionary atom update step).…”