Memory-Augmented Deep Unfolding Network for Compressive Sensing

Abstract

Mapping a truncated optimization method into a deep neural network, deep unfolding network (DUN) has attracted growing attention in compressive sensing (CS) due to its good interpretability and high performance. Each stage in DUNs corresponds to one iteration in optimization. By understanding DUNs from the perspective of the human brain's memory processing, we find there exists two issues in existing DUNs. One is the information between every two adjacent stages, which can be regarded as shortterm memory, is usually lost seriously. The other is no explicit mechanism to ensure that the previous stages affect the current stage, which means memory is easily forgotten. To solve these issues, in this paper, a novel DUN with persistent memory for CS is proposed, dubbed Memory-Augmented Deep Unfolding Network (MADUN). We design a memory-augmented proximal mapping module (MAPMM) by combining two types of memory augmentation mechanisms, namely High-throughput Short-term Memory (HSM) and Cross-stage Long-term Memory (CLM). HSM is exploited to allow DUNs to transmit multi-channel short-term memory, which greatly reduces information loss between adjacent stages. CLM is utilized to develop the dependency of deep information across cascading stages, which greatly enhances network representation capability. Extensive CS experiments on natural and MR images show that with the strong ability to maintain and balance information our MADUN outperforms existing state-ofthe-art methods by a large margin. The source code is available at https://github.com/jianzhangcs/MADUN/.

Model-Guided Multi-Contrast Deep Unfolding Network for MRI Super-resolution Reconstruction

“…As a pioneer work, deep unfolding is first reported in [15], and it designs a learned version of the iterative soft thresholding algorithm (ISTA) that can be unfolded into a neural network form. Since then, a series of works [20,29,33,36,39] demonstrate that deep unfolding methods are applicable to certain optimization algorithms since they can not only optimize the parameters in an end-to-end manner by minimizing the loss function over a large training set, but also integrate model-based and learning-based methods well.…”

PUERT: Probabilistic Under-Sampling and Explicable Reconstruction Network for CS-MRI

“…In [18], a cascaded CNN with a data consistency layer is presented to further ensure measurement fidelity. Most recently, some Deep Unfolding Networks (DUNs) [24]- [30] are developed to integrate the interpretability of traditional model-based approaches and the efficiency of data-driven methods, thus yielding a better recovery performance. Note that DUNs are not limited to CS-MRI, but also widely studied in CS reconstruction [31]- [35].…”

TransCL: Transformer Makes Strong and Flexible Compressive Learning

“…Most of the recent works [12], [13], [14], [15], [16], [17], [18], [19] in the CS community focus on how to reconstruct the original signal, and less attention has been devoted to whether one can perform high-level vision tasks direct in the measurement domain. However, in many applications, such as classification and segmentation, we are not interested in obtaining a precise reconstruction of the scene under view, but rather are only interested in the results of inference tasks.…”