Hyperspectral Snapshot Compressive Imaging with Non-Local Spatial-Spectral Residual Network

Abstract

Snapshot Compressive Imaging is an emerging technology that is based on compressive sensing theory to achieve high-efficiency hyperspectral data acquisition. The core problem of this technology is how to reconstruct 3D hyperspectral data from the 2D snapshot measurement in a fast and high-quality manner. In this paper, we propose a novel deep network, which consists of the symmetric residual module and the non-local spatial-spectral attention module, to learn the reconstruction mapping in a data-driven way. The symmetric residual module uses symmetric residual connections to improve the potential of interaction between convolution operations and further promotes the fusion of local features. The non-local spatial-spectral attention module is designed to capture the non-local spatial-spectral correlation in the hyperspectral image. Specifically, this module calculates the channel attention matrix to capture the global correlations between all of the spectral channels, and it fuses the channel attention attained feature maps and the spatial attention weighted features as the module output, thus both of the spatial-spectral correlations of hyperspectral images can be fully utilized for reconstruction. In addition, a compound loss, including the reconstruction loss, the measurement loss, and the cosine loss, is designed to guide the end-to-end network learning. We experimentally evaluate the proposed method on simulation and real datasets. The experimental results show that the proposed network outperforms the competing methods in terms of the reconstruction quality and running time.

InSPECtor: an end-to-end design framework for compressive pixelated hyperspectral instruments

“…The reconstructor estimates x from knowing y and H. This inversion is not trivial due to the non-uniqueness of the problem, which requires the addition of constraints. Examples of such reconstruction algorithms can be found in [30][31][32] and references therein.…”

Hyperspectral Snapshot Compressive Imaging With Dense Back-Projection Joint Attention Network

“…For hyperspectral snapshot compressive reconstruction [36], Meng et al used the spatial-spectral self-attention (TSA) to process the feature information from the channel dimension and the spatial dimension [30], respectively. Yang et al employed a non-local spatial attention module to capture the long-range dependencies in space, achieving high-quality reconstruction [16]. In recent years, the attention mechanism has been proved to offer great potential in improving the performance of deep convolutional neural networks (CNNs).…”

“…Many works turned to deep learning driven methods and enabled hyperspectral snapshot compressive reconstruction by learning an inverse mapping from snapshot measurements to original HSIs. Some representative methods include ISTA-Net [14], SSR-Net [15], NSSR-Net [16] and so on [17]. Although these deep learning based methods can directly output reconstructions through one-shot feed-forward network computations, their reconstruction quality still needs to be further improved.…”

“…Although these deep learning based methods can directly output reconstructions through one-shot feed-forward network computations, their reconstruction quality still needs to be further improved. Some other works are devoted to developing more complex deep networks to improve reconstruction quality, such as using nonlocal attention [16] and 3D convolution. However, employing complex networks increases the reconstruction time inevitably.…”