Spectral Reconstruction From Satellite Multispectral Imagery Using Convolution and Transformer Joint Network

Du, Dakuan; Gu, Yanfeng; Liu, Tianzhu; Li, Xian

doi:10.1109/tgrs.2023.3285893

Cited by 10 publications

(2 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Liu et al [33] proposed the Swin Transformer algorithm and extensively applied it to the field of computer vision. Furthermore, Du et al [34] designed a Transformer model integrated with convolutional operations, achieving the reconstruction of remote sensing images. However, these methods have been primarily applied to images within the visible light spectrum.…”

Section: Transformermentioning

confidence: 99%

Spectral Reconstruction from Thermal Infrared Multispectral Image Using Convolutional Neural Network and Transformer Joint Network

Zhao,

Qu,

Wang

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

Thermal infrared remotely sensed data, by capturing the thermal radiation characteristics emitted by the Earth’s surface, plays a pivotal role in various domains, such as environmental monitoring, resource exploration, agricultural assessment, and disaster early warning. However, the acquisition of thermal infrared hyperspectral remotely sensed imagery necessitates more complex and higher-precision sensors, which in turn leads to higher research and operational costs. In this study, a novel Convolutional Neural Network (CNN)–Transformer combined block, termed CTBNet, is proposed to address the challenge of thermal infrared multispectral image spectral reconstruction. Specifically, the CTBNet comprises blocks that integrate CNN and Transformer technologies (CTB). Within these CTBs, an improved self-attention mechanism is introduced, which not only considers features across spatial and spectral dimensions concurrently, but also explicitly extracts incremental features from each channel. Compared to other algorithms, the proposed method more closely aligns with the true spectral curves in the reconstruction of hyperspectral images across the spectral dimension. Through a series of experiments, this approach has been proven to ensure robustness and generalizability, outperforming some state-of-the-art algorithms across various metrics.

show abstract

Section: Transformermentioning

confidence: 99%

Spectral Reconstruction from Thermal Infrared Multispectral Image Using Convolutional Neural Network and Transformer Joint Network

Zhao,

Qu,

Wang

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…In addition, a multitemporal spectral reconstruction network (MTSRN) [31] is proposed to reconstruct HS images from multitemporal MS images, which contains a reconstruction network and a temporal features extraction, and a multitemporal fusion network that can independently reconstruct the MS data of a single-phase into HS data and can improve the reconstruction effect by combining neighboring phase information, respectively. Du et al [32] proposed a novel convolution and transformer joint network (CTJN) including cascaded shallow-feature extraction modules (SFEMs) and deep-feature extraction modules (DFEMs), which can explore local spatial features and global spectral features.…”

Section: Spectral Super-resolution Based On Deep Learningmentioning

confidence: 99%

A Multi-Hyperspectral Image Collaborative Mapping Model Based on Adaptive Learning for Fine Classification

Zhang,

Liu,

Zhang

et al. 2024

Remote Sensing

Self Cite

View full text Add to dashboard Cite

Hyperspectral (HS) data, encompassing hundreds of spectral channels for the same area, offer a wealth of spectral information and are increasingly utilized across various fields. However, their limitations in spatial resolution and imaging width pose challenges for precise recognition and fine classification in large scenes. Conversely, multispectral (MS) data excel in providing spatial details for vast landscapes but lack spectral precision. In this article, we proposed an adaptive learning-based mapping model, including an image fusion module, spectral super-resolution network, and adaptive learning network. Spectral super-resolution networks learn the mapping between multispectral and hyperspectral images based on the attention mechanism. The image fusion module leverages spatial and spectral consistency in training data, providing pseudo labels for spectral super-resolution training. And the adaptive learning network incorporates spectral response priors via unsupervised learning, adjusting the output of the super-resolution network to preserve spectral information in reconstructed data. Through the experiment, the model eliminates the need for the manual setting of image prior information and complex parameter selection, and can adjust the network structure and parameters dynamically, eventually enhancing the reconstructed image quality, and enabling the fine classification of large-scale scenes with high spatial resolution. Compared with the recent dictionary learning and deep learning spectral super-resolution methods, our approach exhibits superior performance in terms of both image similarity and classification accuracy.

show abstract

Multifiltering MLP for Spectral Super-Resolution With Remote Sensing Image Verification

Li,

Leng,

Zhang

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Spectral Reconstruction From Satellite Multispectral Imagery Using Convolution and Transformer Joint Network

Cited by 10 publications

References 63 publications

Spectral Reconstruction from Thermal Infrared Multispectral Image Using Convolutional Neural Network and Transformer Joint Network

Spectral Reconstruction from Thermal Infrared Multispectral Image Using Convolutional Neural Network and Transformer Joint Network

A Multi-Hyperspectral Image Collaborative Mapping Model Based on Adaptive Learning for Fine Classification

Multifiltering MLP for Spectral Super-Resolution With Remote Sensing Image Verification

Contact Info

Product

Resources

About