Progressive Spatial–Spectral Joint Network for Hyperspectral Image Reconstruction

Li, Tianshuai; Gu, Yanfeng

doi:10.1109/tgrs.2021.3079969

Cited by 25 publications

(14 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the actual process of HS data acquisition and transformation, external environmental change and internal equipment conditions inevitably lead to noises, blurs, and missing data (including clouds and stripes) [130], [131] which degrade the visual quality of HS images and the efficiency of the subsequent HS data applications, such as a fine HS RS classification for crops and wetlands [132], [133] and the refinement of spectral information for target detection [134], [135]. Fig.…”

Section: Hs Restorationmentioning

confidence: 99%

Tensor Decompositions for Hyperspectral Data Processing in Remote Sensing: A Comprehensive Review

Wang¹,

Hong²,

Han³

et al. 2022

Preprint

View full text Add to dashboard Cite

Owing to the rapid development of sensor technology, hyperspectral (HS) remote sensing (RS) imaging has provided a significant amount of spatial and spectral information for the observation and analysis of the Earth's surface at a distance of data acquisition devices, such as aircraft, spacecraft, and satellite. The recent advancement and even revolution of the HS RS technique offer opportunities to realize the full potential of various applications, while confronting new challenges for efficiently processing and analyzing the enormous HS acquisition data. Due to the maintenance of the 3-D HS inherent structure, tensor decomposition has aroused widespread concern and research in HS data processing tasks over the past decades. In this article, we aim at presenting a comprehensive overview of tensor decomposition, specifically contextualizing the five broad topics in HS data processing, and they are HS restoration, compressed sensing, anomaly detection, superresolution, and spectral unmixing. For each topic, we elaborate on the remarkable achievements of tensor decomposition models for HS RS with a pivotal description of the existing methodologies and a representative exhibition on the experimental results. As a result, the remaining challenges of the follow-up research directions are outlined and discussed from the perspective of the real HS RS practices and tensor decomposition merged with advanced priors and even with deep neural networks. This article summarizes different tensor decomposition-based HS data processing methods and categorizes them into different classes from simple adoptions to complex combinations with other Manuscript received XX, 2022; revised XX, 2022.

show abstract

Section: Hs Restorationmentioning

confidence: 99%

Tensor Decompositions for Hyperspectral Data Processing in Remote Sensing: A Comprehensive Review

Wang¹,

Hong²,

Han³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Earlier researchers adopted the sparse dictionary method [4][5][6][7][8][9]. With the development of deep learning, owing to its excellent feature extraction and reconstruction capabilities, more and more researchers are adopting deep learning methods to gradually replace the traditional sparse dictionary approach [10][11][12][13][14][15][16].…”

Section: Introductionmentioning

confidence: 99%

“…This results in the lack of one essential nir band as the input, which does not make full use of the original information, thereby leading to a waste of information. There are already some studies of remote sensing spectral reconstruction considering this problem [15,16]. Few studies have been conducted on large-scale and highly complex scenarios such as satellite remote sensing.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SRT: A Spectral Reconstruction Network for GF-1 PMS Data Based on Transformer and ResNet

Zhang

Qian

et al. 2022

Remote Sensing

View full text Add to dashboard Cite

The time of acquiring remote sensing data was halved after the joint operation of Gao Fen-6 (GF-6) and Gao Fen-1 (GF-1) satellites. Meanwhile, GF-6 added four bands, including the “red-edge” band that can effectively reflect the unique spectral characteristics of crops. However, GF-1 data do not contain these bands, which greatly limits their application to crop-related joint monitoring. In this paper, we propose a spectral reconstruction network (SRT) based on Transformer and ResNet to reconstruct the missing bands of GF-1. SRT is composed of three modules: (1) The transformer feature extraction module (TFEM) fully extracts the correlation features between spectra. (2) The residual dense module (RDM) reconstructs local features and avoids the vanishing gradient problem. (3) The residual global construction module (RGM) reconstructs global features and preserves texture details. Compared with competing methods, such as AWAN, HRNet, HSCNN-D, and M2HNet, the proposed method proved to have higher accuracy by a margin of the mean relative absolute error (MRAE) and root mean squared error (RMSE) of 0.022 and 0.009, respectively. It also achieved the best accuracy in supervised classification based on support vector machine (SVM) and spectral angle mapper (SAM).

show abstract

“…To promote the development of SSR technology, New Trends in Image Restoration and Enhancement (NTIRE) has organized two SSR competitions in 2018 [35] and 2020 [36]. Numerous deep CNN-based methods have been proposed to learn an end-to-end mapping function between the RGB inputs and the HSI counterparts [29], [28], [37], [31], [38], [30], [39], which greatly intensified the precision of estimated spectral signatures over the previous means based sparse dictionary learning. Although CNN-based methods have made significant breakthroughs in the SSR field, several shortcomings exist in the current models.…”

Section: Introductionmentioning

confidence: 99%

HPRN: Holistic Prior-embedded Relation Network for Spectral Super-Resolution

Wu¹,

Li²,

Song³

et al. 2021

Preprint

View full text Add to dashboard Cite

Spectral super-resolution (SSR) refers to the hyperspectral image (HSI) recovery from an RGB counterpart. Due to the one-to-many nature of the SSR problem, a single RGB image can be reprojected to many HSIs. The key to tackle this illposed problem is to plug into multi-source prior information such as the natural RGB spatial context-prior, deep feature-prior or inherent HSI statistical-prior, etc., so as to improve the confidence and fidelity of reconstructed spectra. However, most current approaches only consider the general and limited priors in their designing the customized convolutional neural networks (CNNs), which leads to the inability to effectively alleviate the degree of ill-posedness. To address the problematic issues, we propose a novel holistic prior-embedded relation network (HPRN) for SSR. Basically, the core framework is delicately assembled by several multi-residual relation blocks (MRBs) that fully facilitate the transmission and utilization of the low-frequency content prior of RGB signals. Innovatively, the semantic prior of RGB input is introduced to identify category attributes and a semantic-driven spatial relation module (SSRM) is put forward to perform the feature aggregation among the clustered similar characteristics using a semantic-embedded relation matrix. Additionally, we develop a transformer-based channel relation module (TCRM), which breaks the habit of employing scalars as the descriptors of channel-wise relations in the previous deep feature-prior and replaces them with certain vectors, together with Transformerstyle feature interactions, supporting the representations to be more discriminative. In order to maintain the mathematical correlation and spectral consistency between hyperspectral bands, the second-order prior constraints (SOPC) are incorporated into the loss function to guide the HSI reconstruction process. Finally, extensive experimental results on four established benchmarks demonstrate that our HPRN can reach the state-of-the-art performance for SSR quantitatively and qualitatively and also, the effectiveness and usefulness of the reconstructed spectrum are verified by the classification results on the remote sensing dataset. Codes are available at https://github.com/Deep-imagelab/HPRN.

show abstract

Progressive Spatial–Spectral Joint Network for Hyperspectral Image Reconstruction

Cited by 25 publications

References 42 publications

Tensor Decompositions for Hyperspectral Data Processing in Remote Sensing: A Comprehensive Review

Tensor Decompositions for Hyperspectral Data Processing in Remote Sensing: A Comprehensive Review

SRT: A Spectral Reconstruction Network for GF-1 PMS Data Based on Transformer and ResNet

HPRN: Holistic Prior-embedded Relation Network for Spectral Super-Resolution

Contact Info

Product

Resources

About