Hyperspectral image super-resolution combining with deep learning and spectral unmixing

Super-resolution (SR) technology has emerged as an effective tool for image analysis and interpretation. However, single hyperspectral (HS) image SR remains challenging, due to the high spectral dimensionality and lack of available high-resolution information of auxiliary sources. To fully exploit the spectral and spatial characteristics, in this paper, a novel single HS image SR approach is proposed based on a spatial correlation-regularized unmixing convolutional neural network (CNN). The proposed approach takes advantage of a CNN to explore the collaborative spatial and spectral information of an HS image and infer the high-resolution abundance maps, thereby reconstructing the anticipated high-resolution HS image via the linear spectral mixture model. Moreover, a dual-branch architecture network and spatial spread transform function are employed to characterize the spatial correlation between the high- and low-resolution HS images, aiming at promoting the fidelity of the super-resolved image. Experiments on three public remote sensing HS images demonstrate the feasibility and superiority in terms of spectral fidelity, compared with some state-of-the-art HS image super-resolution methods.

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Hyperspectral Image Super-Resolution Based on Spatial Correlation-Regularized Unmixing Convolutional Neural Network

Yang

Zhang

et al. 2021

“…The primary technique for improving the spatial resolution of hyperspectral images is image super-resolution (SR) reconstruction. The SR reconstruction algorithms can be classified into three types: image interpolation [21,22], image reconstruction [23], and deep learning-based reconstruction [24][25][26][27]. The image interpolation algorithms are simple, but details are easily lost in the reconstruction results.…”

Section: Introductionmentioning

confidence: 99%

Deep Learning-Based Super-Resolution Reconstruction and Algorithm Acceleration of Mars Hyperspectral CRISM Data

Sun

Chen²

2022

In Mars exploration, hyper-spectrometry plays an important role due to its high spectral resolution. However, due to the technical difficulty and the data size, the spatial resolution or the coverage of hyperspectral data is often limited. This limitation can be alleviated by deep learning-based super-resolution (SR) reconstruction. But the spatial size and batch size of the input training data is limited due to the large number of spectral channels. To improve the efficiency of model training and SR reconstruction, a dataset based on CRISM hyperspectral data is created in this paper, and its redundancy is analyzed in both spectral and spatial spital dimensions. Compression algorithms based on data selection and PCA are used to reduce the size of the input training data. A network that can perform spatial SR and spectral enhancement is also proposed to make the network can be trained with the compressed data. With these compression algorithms and network, high-resolution data with 235 bands can be reconstructed from the low-resolution data with only 40 bands. Compared with the network trained on the original low-resolution data with 235 bands, the model training time and the SR reconstruction runtime can be reduced to 30% and 23% with practically no accuracy loss. The effectiveness of compression algorithms based on data selection also indicates that maybe not all the bands need to be transmitted from the Mars probes or be collected. Furthermore, it would, in principle, help improve the efficiency of satellite data transmission and simplify the design of the hyper-spectrometer. Additionally, a method for spatial dimension correlation evaluation is also proposed in this paper. The spatial compression shows that the proposed method can reflect the correlation of spatial texture between patches, and the model can be acceptably trained with only half of the original data.

“…Thus, spectral unmixing (SU) [1][2][3][4][5] was proposed to decompose the mixed pixels into a set of substance spectra, called endmembers, and their corresponding fractions, called abundances. SU is generally used as a preprocessing step to provide rich pixel features for downstream tasks [6][7][8][9], or can be adopted directly as a method for substance identification [10].…”

Section: Introductionmentioning

confidence: 99%

Gated Autoencoder Network for Spectral–Spatial Hyperspectral Unmixing

Hua

Jiang

et al. 2021

Convolution-based autoencoder networks have yielded promising performances in exploiting spatial–contextual signatures for spectral unmixing. However, the extracted spectral and spatial features of some networks are aggregated, which makes it difficult to balance their effects on unmixing results. In this paper, we propose two gated autoencoder networks with the intention of adaptively controlling the contribution of spectral and spatial features in unmixing process. Gating mechanism is adopted in the networks to filter and regularize spatial features to construct an unmixing algorithm based on spectral information and supplemented by spatial information. In addition, abundance sparsity regularization and gating regularization are introduced to ensure the appropriate implementation. Experimental results validate the superiority of the proposed method to the state-of-the-art techniques in both synthetic and real-world scenes. This study confirms the effectiveness of gating mechanism in improving the accuracy and efficiency of utilizing spatial signatures for spectral unmixing.