Learning Non-Locally Regularized Compressed Sensing Network With Half-Quadratic Splitting

Sun, Yubao; Yang, Ying; Liu, Qingshan; Chen, Jiwei; Yuan, Xin; Guo, Guodong

doi:10.1109/tmm.2020.2973862

Cited by 48 publications

(23 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our proposed ISTA-Net ++ is compared with several representative state-of-the-art methods including BM3D-AMP [18], LDAMP [19], DIP [20], ReconNet [28], DPDNN [14], GDN [15], ISTA-Net + [2], NLR-CSNet [21], DPA-Net [10], and MAC-Net [9]. Different from other deep network-based end-to-end methods, for ISTA-Net ++ , we use the five sampling matrices for five CS ratios to train our model only once.…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

ISTA-NET⁺⁺: Flexible Deep Unfolding Network for Compressive Sensing

You

Xie

Zhang

2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

While deep neural networks have achieved impressive success in image compressive sensing (CS), most of them lack flexibility when dealing with multi-ratio tasks and multiscene images in practical applications. To tackle these challenges, we propose a novel end-to-end flexible ISTAunfolding deep network, dubbed ISTA-Net ++ , with superior performance and strong flexibility. Specifically, by developing a dynamic unfolding strategy, our model enjoys the adaptability of handling CS problems with different ratios, i.e., multi-ratio tasks, through a single model. A cross-block strategy is further utilized to reduce blocking artifacts and enhance the CS recovery quality. Furthermore, we adopt a balanced dataset for training, which brings more robustness when reconstructing images of multiple scenes. Extensive experiments on four datasets show that ISTA-Net ++ achieves state-of-the-art results in terms of both quantitative metrics and visual quality. Considering its flexibility, effectiveness and practicability, our model is expected to serve as a suitable baseline in future CS research.

show abstract

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

ISTA-NET⁺⁺: Flexible Deep Unfolding Network for Compressive Sensing

You

Xie

Zhang

2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

show abstract

“…Specifically, considering the optimizationbased methods, three representative CS schemes are selected, i.e., TV [53], MH [32] and GSR [31]. In view of the deep network-based methods, more than ten CS algorithms are considered, including: SDA [33], ReconNet [16], LDIT [20], LDAMP [20], DPDNN [63], I-Recon [17], ISTA-Net [21], DR 2 -Net [15], IRCNN [42], NN [64], NLR-CSNet [54], DPA-Net [37] and DPIR [44]. b) For the learned sampling matrixbased CS methods, seven recent literatures, i.e., CSNet [55], LapCSNet [35], SCSNet [19], CSNet + [6], BCS-Net [39], OPINE-Net + [23] and AMP-Net [22] participate in the comparison in our experiments.…”

Section: Comparisons With State-of-the-art Methodsmentioning

confidence: 99%

“…3) The proposed non-local module globally explores the self-similar knowledge in the entire image space, which significantly expands the receptive field. Recently, inspired by Deep Image Prior (DIP) [56], Sun et al [54] propose a non-locally regularized CS network, in which the non-local prior and the deep network prior are both considered for image reconstruction. However, for each image, the network in [54] needs to be trained online driven by the two priors in an iterative fashion, which undoubtedly brings about a high computational cost and a lack of flexibility.…”

Section: B Non-local Self-similarity Image Priormentioning

confidence: 99%

“…Recently, inspired by Deep Image Prior (DIP) [56], Sun et al [54] propose a non-locally regularized CS network, in which the non-local prior and the deep network prior are both considered for image reconstruction. However, for each image, the network in [54] needs to be trained online driven by the two priors in an iterative fashion, which undoubtedly brings about a high computational cost and a lack of flexibility. By contrast, the proposed CS network generates final test model through pretraining.…”

Section: B Non-local Self-similarity Image Priormentioning

confidence: 99%

See 1 more Smart Citation

Image Compressed Sensing Using Non-local Neural Network

Cui,

Liu,

Jiang

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep network-based image Compressed Sensing (CS) has attracted much attention in recent years. However, the existing deep network-based CS schemes either reconstruct the target image in a block-by-block manner that leads to serious block artifacts or train the deep network as a black box that brings about limited insights of image prior knowledge. In this paper, a novel image CS framework using non-local neural network (NL-CSNet) is proposed, which utilizes the nonlocal self-similarity priors with deep network to improve the reconstruction quality. In the proposed NL-CSNet, two nonlocal subnetworks are constructed for utilizing the non-local selfsimilarity priors in the measurement domain and the multi-scale feature domain respectively. Specifically, in the subnetwork of measurement domain, the long-distance dependencies between the measurements of different image blocks are established for better initial reconstruction. Analogically, in the subnetwork of multi-scale feature domain, the affinities between the dense feature representations are explored in the multi-scale space for deep reconstruction. Furthermore, a novel loss function is developed to enhance the coupling between the non-local representations, which also enables an end-to-end training of NL-CSNet. Extensive experiments manifest that NL-CSNet outperforms existing state-of-the-art CS methods, while maintaining fast computational speed. The source code of plugging nonlocal module into certain CS network can be found: https: //github.com/WenxueCui/NL-CSNet-Pytorch.

show abstract

“…Network-Driven methods: deep networks have made gratifying progress in visionrelated tasks [15,20,21]. With the help of the excellent representation ability of deep network, some scholars apply it to compressive sensing reconstruction, forming a network-driven reconstruction method.…”

Section: Related Workmentioning

confidence: 99%

Hyperspectral Snapshot Compressive Imaging with Non-Local Spatial-Spectral Residual Network

et al. 2021

Self Cite

View full text Add to dashboard Cite

Snapshot Compressive Imaging is an emerging technology that is based on compressive sensing theory to achieve high-efficiency hyperspectral data acquisition. The core problem of this technology is how to reconstruct 3D hyperspectral data from the 2D snapshot measurement in a fast and high-quality manner. In this paper, we propose a novel deep network, which consists of the symmetric residual module and the non-local spatial-spectral attention module, to learn the reconstruction mapping in a data-driven way. The symmetric residual module uses symmetric residual connections to improve the potential of interaction between convolution operations and further promotes the fusion of local features. The non-local spatial-spectral attention module is designed to capture the non-local spatial-spectral correlation in the hyperspectral image. Specifically, this module calculates the channel attention matrix to capture the global correlations between all of the spectral channels, and it fuses the channel attention attained feature maps and the spatial attention weighted features as the module output, thus both of the spatial-spectral correlations of hyperspectral images can be fully utilized for reconstruction. In addition, a compound loss, including the reconstruction loss, the measurement loss, and the cosine loss, is designed to guide the end-to-end network learning. We experimentally evaluate the proposed method on simulation and real datasets. The experimental results show that the proposed network outperforms the competing methods in terms of the reconstruction quality and running time.

show abstract

Learning Non-Locally Regularized Compressed Sensing Network With Half-Quadratic Splitting

Cited by 48 publications

References 35 publications

ISTA-NET⁺⁺: Flexible Deep Unfolding Network for Compressive Sensing

ISTA-NET⁺⁺: Flexible Deep Unfolding Network for Compressive Sensing

Image Compressed Sensing Using Non-local Neural Network

Hyperspectral Snapshot Compressive Imaging with Non-Local Spatial-Spectral Residual Network

Contact Info

Product

Resources

About

Learning Non-Locally Regularized Compressed Sensing Network With Half-Quadratic Splitting

Cited by 48 publications

References 35 publications

ISTA-NET++: Flexible Deep Unfolding Network for Compressive Sensing

ISTA-NET++: Flexible Deep Unfolding Network for Compressive Sensing

Image Compressed Sensing Using Non-local Neural Network

Hyperspectral Snapshot Compressive Imaging with Non-Local Spatial-Spectral Residual Network

Contact Info

Product

Resources

About

ISTA-NET⁺⁺: Flexible Deep Unfolding Network for Compressive Sensing

ISTA-NET⁺⁺: Flexible Deep Unfolding Network for Compressive Sensing