Multiscale Feature Fusion Network Incorporating 3D Self-Attention for Hyperspectral Image Classification

Qing, Yuhao; Huang, Quanzhen; Feng, Liuyan; Qi, Yueyan; Liu, Wenyi

doi:10.3390/rs14030742

Cited by 20 publications

(12 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, the performance of the HSSF-MLDSVM based hyper-spectral image object classification using HSI has been compared with the other existing feature fusion-based object classification techniques spectral-spatial dependent global learning (SSDGL) [10], central attention network (CAN) [11], convolution neural network -active learning-Markov random field (CNN-Al-MNF) [12], enhanced-multiscale feature-fusion network (EMFFN) [24], 3-dimension self-attention multiscale feature-fusion network (3DSA-MFN) [25], adaptive spectral-spatial feature fusion network (ASSFFN) [26], low-rank attention multiple feature-fusion network (LMAFN) [27], and deep support vector machine (DSVM) [28]. For evaluating the proposed HSSF-MLDSVM and other existing HSI object classification techniques, the Indian Pines dataset has been used.…”

Section: Resultsmentioning

confidence: 99%

“…The fusion method known as enhanced multiscale feature-fusion network (EMFFN) was first introduced in [24]. Using two subnetworks, the spectral cascaded dilated convolutional network (SCDCN) and parallel multipath network (PMN), the model extracts multiscale spatial-spectral information [25]. When extracting multiscale characteristics from long-range data of bigger fields, the SCDCN is employed.…”

Section: Literature Surveymentioning

confidence: 99%

See 1 more Smart Citation

Hyperspectral object classification using hybrid spectral-spatial fusion and noise tolerant soft-margin technique

Mani,

Raguttapalli Chowdareddy

2024

IJECE

View full text Add to dashboard Cite

Because of its spectral-spatial and temporal resolution of greater areas, hyperspectral imaging (HSI) has found widespread application in the field of object classification. The HSI is typically used to accurately determine an object's physical characteristics as well as to locate related objects with appropriate spectral fingerprints. As a result, the HSI has been extensively applied to object identification in several fields, including surveillance, agricultural monitoring, environmental research, and precision agriculture. However, because of their enormous size, objects require a lot of time to classify; for this reason, both spectral and spatial feature fusion have been completed. The existing classification strategy leads to increased misclassification, and the feature fusion method is unable to preserve semantic object inherent features; This study addresses the research difficulties by introducing a hybrid spectral-spatial fusion (HSSF) technique to minimize feature size while maintaining object intrinsic qualities; Lastly, a soft-margins kernel is proposed for multi-layer deep support vector machine (MLDSVM) to reduce misclassification. The standard Indian pines dataset is used for the experiment, and the outcome demonstrates that the HSSF-MLDSVM model performs substantially better in terms of accuracy and Kappa coefficient.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Literature Surveymentioning

confidence: 99%

Hyperspectral object classification using hybrid spectral-spatial fusion and noise tolerant soft-margin technique

Mani,

Raguttapalli Chowdareddy

2024

IJECE

View full text Add to dashboard Cite

show abstract

“…At present, CNN is widely used in the field of computer vision, the convolutional operation only works with local neighborhoods, and it is effective in extracting local features. It is difficult to capture interactive features over long distances, which affects the accuracy of classification to some extent 27 . Therefore, we introduced a transformer.…”

Section: Methodsmentioning

confidence: 99%

“…It is difficult to capture interactive features over long distances, which affects the accuracy of classification to some extent. 27 Therefore, we introduced a transformer. Compared with a CNN, it achieves a significant performance improvement in global feature extraction via a self‐attention mechanism.…”

Section: Methodsmentioning

confidence: 99%

Deep learning‐based prediction of H3K27M alteration in diffuse midline gliomas based on whole‐brain MRI

et al. 2023

View full text Add to dashboard Cite

BackgroundH3K27M mutation status significantly affects the prognosis of patients with diffuse midline gliomas (DMGs), but this tumor presents a high risk of pathological acquisition. We aimed to construct a fully automated model for predicting the H3K27M alteration status of DMGs based on deep learning using whole‐brain MRI.MethodsDMG patients from West China Hospital of Sichuan University (WCHSU; n = 200) and Chengdu Shangjin Nanfu Hospital (CSNH; n = 35) who met the inclusion and exclusion criteria from February 2016 to April 2022 were enrolled as the training and external test sets, respectively. To adapt the model to the human head MRI scene, we use normal human head MR images to pretrain the model. The classification and tumor segmentation tasks are naturally related, so we conducted cotraining for the two tasks to enable information interaction between them and improve the accuracy of the classification task.ResultsThe average classification accuracies of our model on the training and external test sets was 90.5% and 85.1%, respectively. Ablation experiments showed that pretraining and cotraining could improve the prediction accuracy and generalization performance of the model. In the training and external test sets, the average areas under the receiver operating characteristic curve (AUROCs) were 94.18% and 87.64%, and the average areas under the precision‐recall curve (AUPRC) were 93.26% and 85.4%.ConclusionsThe developed model achieved excellent performance in predicting the H3K27M alteration status in DMGs, and its good reproducibility and generalization were verified in the external dataset.

show abstract

“…Yu et al [35] proposed an image-based global learning framework of a dual-channel convolutional network (DCCN) that optimizes the utilization of global and multiscale information for HSI classification. Qing et al [36] introduced a 3D self-attention multiscale feature fusion network (3DSA-MFN) for HSI classification, incorporating 3D multihead self-attention to capture interactive features over long distances and effectively fuse spatial and spectral features. Zhong et al [37] introduced a spectral space transform network (SSTN), with spatial attention and spectral correlation modules, and a factorized architecture search (FAS) framework for hyperspectral image classification.…”

Section: Hyperspectral Image Classificationmentioning

confidence: 99%

S3L: Spectrum Transformer for Self-Supervised Learning in Hyperspectral Image Classification

Guo,

Liu

2024

Remote Sensing

Self Cite

View full text Add to dashboard Cite

In the realm of Earth observation and remote sensing data analysis, the advancement of hyperspectral imaging (HSI) classification technology is of paramount importance. Nevertheless, the intricate nature of hyperspectral data, coupled with the scarcity of labeled data, presents significant challenges in this domain. To mitigate these issues, we introduce a self-supervised learning algorithm predicated on a spectral transformer for HSI classification under conditions of limited labeled data, with the objective of enhancing the efficacy of HSI classification. The S3L algorithm operates in two distinct phases: pretraining and fine-tuning. During the pretraining phase, the algorithm learns the spatial representation of HSI from unlabeled data, utilizing a masking mechanism and a spectral transformer, thereby augmenting the sequence dependence of spectral features. Subsequently, in the fine-tuning phase, labeled data is employed to refine the pretrained weights, thereby improving the precision of HSI classification. Within the comprehensive encoder–decoder framework, we propose a novel spectral transformer module specifically engineered to synergize spatial feature extraction with spectral domain analysis. This innovative module adeptly navigates the complex interplay among various spectral bands, capturing both global and sequential spectral dependencies. Uniquely, it incorporates a gated recurrent unit (GRU) layer within the encoder to enhance its ability to process spectral sequences. Our experimental evaluations across several public datasets reveal that our proposed method, distinguished by its spectral transformer, achieves superior classification performance, particularly in scenarios with limited labeled samples, outperforming existing state-of-the-art approaches.

show abstract

Multiscale Feature Fusion Network Incorporating 3D Self-Attention for Hyperspectral Image Classification

Cited by 20 publications

References 68 publications

Hyperspectral object classification using hybrid spectral-spatial fusion and noise tolerant soft-margin technique

Hyperspectral object classification using hybrid spectral-spatial fusion and noise tolerant soft-margin technique

Deep learning‐based prediction of H3K27M alteration in diffuse midline gliomas based on whole‐brain MRI

S3L: Spectrum Transformer for Self-Supervised Learning in Hyperspectral Image Classification

Contact Info

Product

Resources

About