Online Adaptive Image Reconstruction (OnAIR) Using Dictionary Models

Abstract

Sparsity and low-rank models have been popular for reconstructing images and videos from limited or corrupted measurements. Dictionary or transform learning methods are useful in applications such as denoising, inpainting, and medical image reconstruction. This paper proposes a framework for online (or time-sequential) adaptive reconstruction of dynamic image sequences from linear (typically undersampled) measurements. We model the spatiotemporal patches of the underlying dynamic image sequence as sparse in a dictionary, and we simultaneously estimate the dictionary and the images sequentially from streaming measurements. Multiple constraints on the adapted dictionary are also considered such as a unitary matrix, or low-rank dictionary atoms that provide additional efficiency or robustness. The proposed online algorithms are memory efficient and involve simple updates of the dictionary atoms, sparse coefficients, and images. Numerical experiments demonstrate the usefulness of the proposed methods in inverse problems such as video reconstruction or inpainting from noisy, subsampled pixels, and dynamic magnetic resonance image reconstruction from very limited measurements.Index Terms-Online methods, sparse representations, dictionary learning, machine learning, inverse problems, video processing, dynamic magnetic resonance imaging.

Image Reconstruction: From Sparsity to Data-Adaptive Methods and Machine Learning

“…can be learned in various ways such as by using training datasets [25], [26], or even learned jointly with the reconstruction [25], [27]- [29], a setting called model-blind reconstruction or blind compressed sensing (BCS) [30]. While most of these methods perform offline reconstruction (where the reconstruction is performed once all the measurements are collected), recent works show that the models can also be learned in a time-sequential or online manner from streaming measurements to reconstruct dynamic objects [31], [32]. The learning can be done in an unsupervised manner employing model-based and surrogate cost functions, or the reconstruction algorithms (such as deep convolutional neural networks (CNNs)) can be trained in a supervised manner to minimize the error in reconstructing training datasets that typically consist of pairs of ground truth and undersampled data [33]- [37].…”

“…Recent works have proposed online learning of sophisticated models for reconstruction particularly of dynamic data from time-series measurements [32], [143], [144]. In this setting, the reconstructions are produced in a time-sequential manner from the incoming measurement sequence, with the models also adapted simultaneously and sequentially over time to track the underlying object's dynamics and aid reconstruction.…”

“…A recent work efficiently adapted low-rank tensor models in an online manner for dynamic MRI [144]. Online learning for dynamic image reconstruction was shown to be promising in [32], [143], which adapted synthesis dictionaries to spatio-temporal patches. In this setup [32], measurements corresponding to a group (called mini-batch) of frames are processed at a time using a sliding window strategy.…”

“…Online learning for dynamic image reconstruction was shown to be promising in [32], [143], which adapted synthesis dictionaries to spatio-temporal patches. In this setup [32], measurements corresponding to a group (called mini-batch) of frames are processed at a time using a sliding window strategy. The objective function for reconstruction is a weighted time average of instantaneous cost functions, each corresponding to a group of processed frames.…”

“…Each frame can be reconstructed from multiple overlapping temporal windows and a weighted average of those used as the final estimate. The online learning algorithms in [32] achieved computational efficiency by using warm start initializations (that improve over time) for variables and frames based on estimates in previous windows, and thus running only a few iterations of optimization for each new window. They stored past information in small (cumulatively updated) matrices for the dictionary update (low memory usage).…”