DSSFN: A Dual-Stream Self-Attention Fusion Network for Effective Hyperspectral Image Classification

Zi-an, Yang; Zheng, Nai-Rong; Wang, Feng

doi:10.3390/rs15153701

Cited by 4 publications

(6 citation statements)

References 84 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unlike the former models, similarity-based attention modules [150,187,188,190,191,193,200,205,211] measure the spectral similarity between pixels to decide the importance of each pixel. The classic SA modules [145], which were used to locate the crucial words of sentences in the field of neural machine translation, adopted the dot-product similarity to evaluate the spectral correlations between all pixels [180][181][182]. The generation of spatial attention was actually an operation on query, key, and value sets.…”

Section: Similarity-based Spatial Attentionmentioning

confidence: 99%

“…The SSP models based on feature fusion integrate spectral and spatial features in different ways, e.g., concatenation and addition, before classification [77,78,80,90,111,119,131,153,161,182,186]. Addition was exploited to aggregate the spectral and spatial features in different modes and keep the consistency of shapes.…”

Section: Spectral-spatial In Parallel (Ssp) Modelsmentioning

confidence: 99%

“…The classic structures contain SA [145] and non-local [148] modules. They obtained the dot-product spectral similarity between all pixels for normalized weights [180][181][182][183]. They have been applied to highlight vital pixels in many architectures [184][185][186].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Discriminating Spectral–Spatial Feature Extraction for Hyperspectral Image Classification: A Review

Li,

Wang,

Cheikh

2024

Sensors

View full text Add to dashboard Cite

Hyperspectral images (HSIs) contain subtle spectral details and rich spatial contextures of land cover that benefit from developments in spectral imaging and space technology. The classification of HSIs, which aims to allocate an optimal label for each pixel, has broad prospects in the field of remote sensing. However, due to the redundancy between bands and complex spatial structures, the effectiveness of the shallow spectral–spatial features extracted by traditional machine-learning-based methods tends to be unsatisfying. Over recent decades, various methods based on deep learning in the field of computer vision have been proposed to allow for the discrimination of spectral–spatial representations for classification. In this article, the crucial factors to discriminate spectral–spatial features are systematically summarized from the perspectives of feature extraction and feature optimization. For feature extraction, techniques to ensure the discrimination of spectral features, spatial features, and spectral–spatial features are illustrated based on the characteristics of hyperspectral data and the architecture of models. For feature optimization, techniques to adjust the feature distances between classes in the classification space are introduced in detail. Finally, the characteristics and limitations of these techniques and future challenges in facilitating the discrimination of features for HSI classification are also discussed further.

show abstract

Section: Similarity-based Spatial Attentionmentioning

confidence: 99%

Section: Spectral-spatial In Parallel (Ssp) Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Discriminating Spectral–Spatial Feature Extraction for Hyperspectral Image Classification: A Review

Li,

Wang,

Cheikh

2024

Sensors

View full text Add to dashboard Cite

show abstract

“…The size of the input network spectral dimension is affected by the channel number, and the size of spatial input sizes determines the amount of spatial information and redundant information [40]. The fitting degree and processing speed of the model are determined by the size of the dropout [41]. From Figures 12-15, the optimal values of the hyperparameters for different hyperspectral datasets are different.…”

Section: Optimal Hyperparameters For Hsismentioning

confidence: 99%

Hybrid Convolutional Network Combining Multiscale 3D Depthwise Separable Convolution and CBAM Residual Dilated Convolution for Hyperspectral Image Classification

Hu,

Tian,

2023

Remote Sensing

View full text Add to dashboard Cite

In recent years, convolutional neural networks (CNNs) have been increasingly leveraged for the classification of hyperspectral imagery, displaying notable advancements. To address the issues of insufficient spectral and spatial information extraction and high computational complexity in hyperspectral image classification, we introduce the MDRDNet, an integrated neural network model. This novel architecture is comprised of two main components: a Multiscale 3D Depthwise Separable Convolutional Network and a CBAM-augmented Residual Dilated Convolutional Network. The first component employs depthwise separable convolutions in a 3D setting to efficiently capture spatial–spectral characteristics, thus substantially reducing the computational burden associated with 3D convolutions. Meanwhile, the second component enhances the network by integrating the Convolutional Block Attention Module (CBAM) with dilated convolutions via residual connections, effectively counteracting the issue of model degradation. We have empirically evaluated the MDRDNet’s performance by running comprehensive experiments on three publicly available datasets: Indian Pines, Pavia University, and Salinas. Our findings indicate that the overall accuracy of the MDRDNet on the three datasets reached 98.83%, 99.81%, and 99.99%, respectively, which is higher than the accuracy of existing models. Therefore, the MDRDNet proposed in this study can fully extract spatial–spectral joint information, providing a new idea for solving the problem of large model calculations in 3D convolutions.

show abstract

“…Early fusion approaches [1] typically combine raw or pre-processed sensor data from different modalities, but they are often susceptible to spatial or temporal misalignment. Conversely, late fusion approaches [2,3] integrate data from various modalities at the decision level, offering greater flexibility to incorporate novel sensing modalities into the network. However, late fusion approaches fail to fully exploit the potential of available sensing modalities as they do not leverage the intermediate features acquired through joint representation learning.…”

Section: Introductionmentioning

confidence: 99%

Deep LiDAR-Radar-Visual Fusion for Object Detection in Urban Environments

Xiao,

Liu,

Luan

et al. 2023

Remote Sensing

View full text Add to dashboard Cite

Robust environmental sensing and accurate object detection are crucial in enabling autonomous driving in urban environments. To achieve this goal, autonomous mobile systems commonly integrate multiple sensor modalities onboard, aiming to enhance accuracy and robustness. In this article, we focus on achieving accurate 2D object detection in urban autonomous driving scenarios. Considering the occlusion issues of using a single sensor from a single viewpoint, as well as the limitations of current vision-based approaches in bad weather conditions, we propose a novel multi-modal sensor fusion network called LRVFNet. This network effectively combines data from LiDAR, mmWave radar, and visual sensors through a deep multi-scale attention-based architecture. LRVFNet comprises three modules: a backbone responsible for generating distinct features from various sensor modalities, a feature fusion module utilizing the attention mechanism to fuse multi-modal features, and a pyramid module for object reasoning at different scales. By effectively fusing complementary information from multi-modal sensory data, LRVFNet enhances accuracy and robustness in 2D object detection. Extensive evaluations have been conducted on the public VOD dataset and the Flow dataset. The experimental results demonstrate the superior performance of our proposed LRVFNet compared to state-of-the-art baseline methods.

show abstract

DSSFN: A Dual-Stream Self-Attention Fusion Network for Effective Hyperspectral Image Classification

Cited by 4 publications

References 84 publications

Discriminating Spectral–Spatial Feature Extraction for Hyperspectral Image Classification: A Review

Discriminating Spectral–Spatial Feature Extraction for Hyperspectral Image Classification: A Review

Hybrid Convolutional Network Combining Multiscale 3D Depthwise Separable Convolution and CBAM Residual Dilated Convolution for Hyperspectral Image Classification

Deep LiDAR-Radar-Visual Fusion for Object Detection in Urban Environments

Contact Info

Product

Resources

About