iPiano-Net: Nonconvex optimization inspired multi-scale reconstruction network for compressed sensing

Su, Yueming; Lian, Qiusheng

doi:10.1016/j.image.2020.115989

Cited by 26 publications

(9 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DUNs on CS and compressive sensing MRI (CS-MRI) usually integrate some effective convolutional neural network (CNN) denoisers into some optimization methods including half quadratic splitting (HQS) [46][5] [1], alternating minimization (AM) [26][31] [53], iterative shrinkage-thresholding algorithm (ISTA) [40][8] [41], approximate message passing (AMP) [48][54], alternating direction method of multipliers (ADMM) [37] and inertial proximal algorithm for nonconvex optimization (iPiano) [30]. Different optimization methods usually lead to different optimization-inspired DUNs.…”

Section: Deep Unfolding Networkmentioning

confidence: 99%

“…Each image block of size 33 × 33 is sampled and reconstructed independently for the first 400 epochs, and for the last ten epochs, we adopt larger image blocks of size 99 × 99 as inputs to further fine-tune the model. To alleviate blocking artifacts, we firstly unfold the blocks of size 99×99 into overlapping blocks of size 33×33 while sampling process Φx and then fold the blocks of size 33 × 33 into larger blocks while initialization Φ ⊤ y [30]. We also unfold the whole image with this approach during testing.…”

Section: Experiments 41 Implementation Detailsmentioning

confidence: 99%

“…3. The models of ReconNet [16], ISTA-Net + [40], DPA-Net [32], IRCNN [46], MAC-Net [2] and iPiano-Net [30] are trained by the same methods and training datasets with the corresponding works, and DPDNN [5] and GDN [8] utilize the same training dataset with our method due to no CS reconstruction task in their original works. One can observe that our MADUN outperforms all the other competing methods in PSNR and SSIM across all the cases.…”

Section: Qualitative Evaluationmentioning

confidence: 99%

See 2 more Smart Citations

Memory-Augmented Deep Unfolding Network for Compressive Sensing

Song

Chen

Zhang

2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Mapping a truncated optimization method into a deep neural network, deep unfolding network (DUN) has attracted growing attention in compressive sensing (CS) due to its good interpretability and high performance. Each stage in DUNs corresponds to one iteration in optimization. By understanding DUNs from the perspective of the human brain's memory processing, we find there exists two issues in existing DUNs. One is the information between every two adjacent stages, which can be regarded as shortterm memory, is usually lost seriously. The other is no explicit mechanism to ensure that the previous stages affect the current stage, which means memory is easily forgotten. To solve these issues, in this paper, a novel DUN with persistent memory for CS is proposed, dubbed Memory-Augmented Deep Unfolding Network (MADUN). We design a memory-augmented proximal mapping module (MAPMM) by combining two types of memory augmentation mechanisms, namely High-throughput Short-term Memory (HSM) and Cross-stage Long-term Memory (CLM). HSM is exploited to allow DUNs to transmit multi-channel short-term memory, which greatly reduces information loss between adjacent stages. CLM is utilized to develop the dependency of deep information across cascading stages, which greatly enhances network representation capability. Extensive CS experiments on natural and MR images show that with the strong ability to maintain and balance information our MADUN outperforms existing state-ofthe-art methods by a large margin. The source code is available at https://github.com/jianzhangcs/MADUN/.

show abstract

Section: Deep Unfolding Networkmentioning

confidence: 99%

Section: Experiments 41 Implementation Detailsmentioning

confidence: 99%

Section: Qualitative Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Memory-Augmented Deep Unfolding Network for Compressive Sensing

Song

Chen

Zhang

2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…Deep unfolding models denote a series of models constructed by mapping iterative algorithms with unfixed numbers of steps onto deep neural networks with fixed numbers of steps [26], [8], [19], [9]. There are a lot of non-linear iterative algorithms that are unfolded, such as ISTA [27], [8], AMP [26], [9], half-quadratic splitting (HQS) [19], alternating direction methods of multipliers (ADMM) [28], [29] and iPiano algorithm [30]. By combining the interpretability of model-based methods and the trainable characteristics of traditional deep learning models, they make a good balance between reconstruction performance and interpretation.…”

Section: Introductionmentioning

confidence: 99%

“…At present, there exist a few methods [32], [30], [33], [34], [17] which reconstruct images at different CS ratios using only one model, and they can be roughly cast into two categories. The first kind [32], [30] trains a single model with a set of sampling matrices with different CS ratios so that the model can adapt to all sampling matrices in this set.…”

Section: Introductionmentioning

confidence: 99%

Scalable Deep Compressive Sensing

Zhang¹,

Liu²,

Cao³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep learning has been used to image compressive sensing (CS) for enhanced reconstruction performance. However, most existing deep learning methods train different models for different subsampling ratios, which brings additional hardware burden. In this paper, we develop a general framework named scalable deep compressive sensing (SDCS) for the scalable sampling and reconstruction (SSR) of all existing end-to-end-trained models. In the proposed way, images are measured and initialized linearly. Two sampling masks are introduced to flexibly control the subsampling ratios used in sampling and reconstruction, respectively. To make the reconstruction model adapt to any subsampling ratio, a training strategy dubbed scalable training is developed. In scalable training, the model is trained with the sampling matrix and the initialization matrix at various subsampling ratios by integrating different sampling matrix masks. Experimental results show that models with SDCS can achieve SSR without changing their structure while maintaining good performance, and SDCS outperforms other SSR methods.

show abstract