Hyperspectral Image Classification Based on Multi-Level Spectral-Spatial Transformer Network

Yang, Hao; Yu, Haoyang; Hong, Danfeng; Xu, Zhen; Wang, Yulei; Song, Ming

doi:10.1109/whispers56178.2022.9955116

Cited by 13 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Transformer model is also used in the image field. It can help improve the performance of image classification [26,27], contribute to generating more realistic pictures [28], assist in object detection [29], and play a critical role in semantic segmentation [30].…”

Section: Transformer Modulementioning

confidence: 99%

Enhancing Intrinsic Image Decomposition with Transformer and Laplacian Pyramid Network

Liu,

Ma,

Meng

et al. 2024

View full text Add to dashboard Cite

Intrinsic Image Decomposition (IID) remains a pivotal challenge in the domain of computer vision, with applications spanning image editing, color image denoising, and segmentation, among others. Despite notable successes, there exists a significant opportunity for enhancing the feature encoding process to improve the accuracy of predicted outcomes. In response to this, a novel framework, termed Transformer and Laplacian Pyramid Network (TLPNet), is introduced. TLPNet comprises two distinct sub-networks: the Transformer for Reflectance Network (TRNet) and the Laplacian Pyramid for Shading Network (LPSNet). Within this framework, the Transformer module is strategically employed within the reflectance imaging component to effectively address the challenge of inadequate feature information extraction. Comprehensive experiments conducted on the ShapeNet Dataset and MIT Dataset have demonstrated the efficacy of TLPNet in predicting more accurate reflectance and shading images. This study contributes to the field by presenting an innovative approach that leverages the strengths of transformer models and Laplacian pyramid structures for the task of IID, setting a new benchmark for future research in the area.

show abstract

Section: Transformer Modulementioning

confidence: 99%

Enhancing Intrinsic Image Decomposition with Transformer and Laplacian Pyramid Network

Liu,

Ma,

Meng

et al. 2024

View full text Add to dashboard Cite

show abstract

“…The utilization of deep learning methodologies has yielded commendable results, exemplified by Contextual-CNN [5], DBN [6], and HybridSN [7]. The integration of transformer models into the realm of image processing has introduced remarkable advancements in hyperspectral image classification [8], delivering heightened precision as demonstrated by SST [9] and SpectralFormer [10], both of which leverage the transformer mechanism. Nonetheless, due to inherent limitations within the data itself, hyperspectral images often confront challenges such as homospectral foreign objects and anomalies, coupled with susceptibility to environmental factors during collection, subsequently constraining their practical applicability [11].…”

Section: Introductionmentioning

confidence: 99%

Multisource remote sensing image classification based on adaptive pooling transformer and multilevel correction

Wang,

Shi,

Gao

et al. 2024

Fifteenth International Conference on Graphics and Image Processing (ICGIP 2023)

View full text Add to dashboard Cite

Deep learning and neural networks have ascended to prominence as the prevailing methodologies for the computational manipulation of remote sensing imagery. The collection and utilization of single remote sensing data inevitably encounter limitations, such as the arduous task of subdividing akin characteristics within hyperspectral data. Endeavors to collaborate with multiple remote sensing data sources are intensifying, aiming to attain more refined and comprehensive observational outcomes. In this article, we propose a comprehensive system framework for the processing of multi-source remote sensing image classification, named the adaptive pooling transformer network (APTnet). Primarily, Convolutional Neural Networks (CNNs) are harnessed to extract distinctive attributes from both hyperspectral images (HSI) and synthetic aperture radar images (SAR). Furthermore, we propose a technique for the fusion of remote sensing data, known as the cross-modal correction fusion method (CMCF), enabling interactive learning and rectification of multi-source features across diverse strata. Additionally, an adaptive pooling Transformer method (APT) is put forth to fortify the existing attributes and amplify their capacity for expression. The experiments conducted on the Berlin and Augsburg datasets exhibit commendable performance, complemented by exhaustive comparative analyses with prevailing methodologies.

show abstract

“…Transformer [15] has a strong ability to capture long-term and short-term information, and has achieved brilliant achievements in computer vision tasks, including image classification [16][17][18], image deraining [19], object detection [20,21], low-level image processing [22,23], action recognition [24][25][26], and other fields. However, these methods tend to require a large amount of GPU memory, thereby hindering the further advancement of Transformer.…”

Section: Introductionmentioning

confidence: 99%

Single‐image super‐resolution using lightweight transformer‐convolutional neural network hybrid model

Liu

Yue

Han

et al. 2023

IET Image Processing

View full text Add to dashboard Cite

With constant advances in deep learning methods as applied to image processing, deep convolutional neural networks (CNNs) have been widely explored in single‐image super‐resolution (SISR) problems and have attained significant success. These CNN‐based methods cannot fully use the internal and external information of the image. The authors add a lightweight Transformer structure to capture this information. Specifically, the authors apply a dense block structure and residual connection to build a residual dense convolution block (RDCB) that reduces the parameters somewhat and extracts shallow features. The lightweight transformer block (LTB) further extracts features and learns the texture details between the patches through the self‐attention mechanism. The LTB comprises an efficient multi‐head transformer (EMT) with small graphics processing unit (GPU) memory footprint, and benefits from feature preprocessing by multi‐head attention (MA), reduction, and expansion. The EMT significantly reduces the use of GPU resources. In addition, a detail‐purifying attention block (DAB) is proposed to explore the context information in the high‐resolution (HR) space to recover more details. Extensive evaluations of four benchmark datasets demonstrate the effectiveness of the authors’ proposed model in terms of quantitative metrics and visual effects. The proposed EMT only uses about 40% as much GPU memory as other methods, with better performance.

show abstract

Hyperspectral Image Classification Based on Multi-Level Spectral-Spatial Transformer Network

Cited by 13 publications

References 11 publications

Enhancing Intrinsic Image Decomposition with Transformer and Laplacian Pyramid Network

Enhancing Intrinsic Image Decomposition with Transformer and Laplacian Pyramid Network

Multisource remote sensing image classification based on adaptive pooling transformer and multilevel correction

Single‐image super‐resolution using lightweight transformer‐convolutional neural network hybrid model

Contact Info

Product

Resources

About