Image Compressed Sensing Using Non-Local Neural Network

Cui, Wen‐Xue; Liu, Shaohui; Jiang, Feng; Zhao, Debin

doi:10.1109/tmm.2021.3132489

Cited by 52 publications

(22 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To tackle this problem, one approach is to use the underlying image priors (e.g., sparsity [5], deep image prior [8] and NSS [7]) to design effective regularizers, which can play a critical role in ensuring accurate image reconstruction. In the past decade, numerous prior-based models have been proposed for image CS, including mainly two categories: local image prior-based models [2,4,9] and nonlocal image prior-based models [6,12,14]. In this section, we review some of the approaches in image CS (reconstruction from limited data) based on local image priors, followed by recent advances in nonlocal image CS reconstruction.…”

Section: Image Cs: From Local To Nonlocal Methodsmentioning

confidence: 99%

“…From the perspective of models, TV [4] and JASR [23] methods are based on analytical models, while other methods are learning-based models. DL-CS [2] CSR [18] CS-GSR [12] JASR [23] GSRC [7] GSC-NCR [13] HSSE [24] CS-NLR [6] RRC [26] LR-GSC [17] CS-Net [9] NL-CSNet [14] CS-GSR [12] and LR-GSC [17] methods both integrate GSR and LR models. From the perspective of model properties, deep learning methods (i.e., CS-Net [9] and NL-CSNet [14]) are supervised learning schemes, while other methods are self-supervised learning schemes.…”

Section: Qualitative Comparison Of Different Image Cs Methodsmentioning

confidence: 99%

“…Beyond employing different NSS priors by learning, the low-rank regularized group sparse coding (LR-GSC) model [17] integrates different GSR and LR learning formulations into a unified CS framework by reconciling nonlocal image modeling. The most recent trend combined deep neural networks with NSS prior for image CS, such as learning nonlocal regularized compressed sensing network (NLR-CSNet) [27], and image CS framework using nonlocal neural network (NL-CSNet) [14]. Such nonlocal model-based supervised deep learning image CS schemes have also yielded promising performance.…”

Section: B Nonlocal Image Csmentioning

confidence: 99%

See 2 more Smart Citations

Learning Nonlocal Sparse and Low-Rank Models for Image Compressive Sensing

Zha¹,

Wen²,

Yuan³

et al. 2022

Preprint

View full text Add to dashboard Cite

The compressive sensing (CS) scheme exploits much fewer measurements than suggested by the Nyquist-Shannon sampling theorem to accurately reconstruct images, which has attracted considerable attention in the computational imaging community. While classic image CS schemes employ sparsity using analytical transforms or bases, the learning-based approaches have become increasingly popular in recent years. Such methods can effectively model the structures of image patches by optimizing their sparse representations or learning deep neural networks, while preserving the known or modeled sensing process. Beyond exploiting local image properties, advanced CS schemes adopt nonlocal image modeling, by extracting similar or highly correlated patches at different locations of an image to form a group to process jointly. More recent learning-based CS schemes apply nonlocal structured sparsity prior using group sparse (and related) representation (GSR) and/or low-rank (LR) modeling, which have demonstrated promising performance in various computational imaging and image processing applications. This article reviews some recent works in image CS tasks with a focus on the advanced GSR and LR based methods. Furthermore, we present a unified framework for incorporating various GSR and LR models and discuss the relationship between GSR and LR models. Finally, we discuss the open problems and future directions in the field.

show abstract

Section: Image Cs: From Local To Nonlocal Methodsmentioning

confidence: 99%

Section: Qualitative Comparison Of Different Image Cs Methodsmentioning

confidence: 99%

Section: B Nonlocal Image Csmentioning

confidence: 99%

See 1 more Smart Citation

Learning Nonlocal Sparse and Low-Rank Models for Image Compressive Sensing

Zha¹,

Wen²,

Yuan³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Examples of such approaches include orthogonal matching pursuit (OMP) [ 10 ], basis pursuit (BP) [ 32 ], the iterative shrinkage thresholding algorithm (ISTA) [ 33 ] and the alternating direction method of multipliers (ADMM) [ 34 ]. To further enhance recovery performance, researchers established more detailed structures based on wavelet tree sparsity [ 35 ], non-local information [ 36 ], minimal total variation [ 37 ] and simple representations in adaptive bases [ 38 ]. However, these conventional CS approaches are usually afflicted with high computational complexity caused by hundreds of iterations.…”

Section: Related Workmentioning

confidence: 99%

A Fast Multi-Scale Generative Adversarial Network for Image Compressed Sensing

Zhu

et al. 2022

Entropy

View full text Add to dashboard Cite

Recently, deep neural network-based image compressed sensing methods have achieved impressive success in reconstruction quality. However, these methods (1) have limitations in sampling pattern and (2) usually have the disadvantage of high computational complexity. To this end, a fast multi-scale generative adversarial network (FMSGAN) is implemented in this paper. Specifically, (1) an effective multi-scale sampling structure is proposed. It contains four different kernels with varying sizes so that decompose, and sample images effectively, which is capable of capturing different levels of spatial features at multiple scales. (2) An efficient lightweight multi-scale residual structure for deep image reconstruction is proposed to balance receptive field size and computational complexity. The key idea is to apply smaller convolution kernel sizes in the multi-scale residual structure to reduce the number of operations while maintaining the receptive field. Meanwhile, the channel attention structure is employed for enriching useful information. Moreover, perceptual loss is combined with MSE loss and adversarial loss as the optimization function to recover a finer image. Numerous experiments show that our FMSGAN achieves state-of-the-art image reconstruction quality with low computational complexity.

show abstract

“…Thus, some works, such as ISTA-Net [27], combine traditional methods with neural networks by replacing the linear or nonlinear steps in each iteration of a traditional method with designed CNN units. Cui and Sun et al respectively use the non-local self-similarity prior in the measurement domain and the multi-scale feature domain to find similar vectors in the size-limited vector space, fill each other with the missing information, and reconstruct the original image [28], [29]. In the paper [30], the rank residual minimization algorithm is combined with deep network units to obtain highly competitive reconstruction results.…”

Section: Introductionmentioning

confidence: 99%

Meta-TR: Meta-Attention Spatial Compressive Imaging Network With Swin Transformer

Cui

Yang

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

As a flourishing research topic in the field of remote sensing, SCI (spatial compressive imaging) can utilize prior knowledge to recover high-dimensional signals from LR (lowresolution) measurements through joint sampling and compression, thus contributing to the bandwidth reduction of information transmission. However, most of the existing SCI methods based on deep learning cannot effectively utilize prior information, and difficult to perform deep extraction of image features, so the reconstruction is not ideal in the case of low sampling ratio. To address the above difficulty, we propose a SCI network based on MA (meta-attention) and swin transformer, named Meta-TR. We adopt the swin transformer as the network backbone, through the wide application of self-attention mechanisms, to achieve deeper extraction of image features, thereby improving the reconstruction quality under low sampling ratios. In addition, we design a meta-attention module, which adopts Squeeze-Excitation architecture to convert the metadata of SCI image degradation process to attention vectors. Then the attention vectors are used in the channel modulation of network feature maps to guide the network training. Extensive experiments are performed on different benchmark remote sensing datasets and different sampling ratios to confirm the superiority of the proposed Meta-TR method.

show abstract

Image Compressed Sensing Using Non-Local Neural Network

Cited by 52 publications

References 64 publications

Learning Nonlocal Sparse and Low-Rank Models for Image Compressive Sensing

Learning Nonlocal Sparse and Low-Rank Models for Image Compressive Sensing

A Fast Multi-Scale Generative Adversarial Network for Image Compressed Sensing

Meta-TR: Meta-Attention Spatial Compressive Imaging Network With Swin Transformer

Contact Info

Product

Resources

About