MSLAENet: Multiscale Learning and Attention Enhancement Network for Fusion Classification of Hyperspectral and LiDAR Data

Abstract

The effective use of multimodal data to obtain accurate land cover information has become an interesting and challenging research topic in the field of remote sensing. In this paper, we propose a new method, multi-scale learning and attention enhancement network (MSLAENet), to implement Hyperspectral image (HSI) and light detection and ranging (LiDAR) data fusion classification in an end-to-end manner. Specifically, our model consists of three main modules. First, we design the composite attention (CA) module, which adopts self-attention to enhance the feature representations of HSI and LiDAR data, respectively, and cross-attention to achieve crossmodal information enhancement. Second, the proposed multiscale learning (MSL) module combines self-calibrated convolutions and hierarchical residual structure to extract different scales of information to further improve the representation capability of the model. Finally, the attention-based feature fusion (FF) module fully considers the complementary information properties between different modalities and adaptively fuses heterogeneous features from different modalities. To test the performance of MSLAENet, we conduct experiments on three multimodal remote sensing datasets and compare them with the state-ofthe-art fusion model, which demonstrates the effectiveness and superiority of the model.

Faster and Lighter Meteorological Satellite Image Classification by a Lightweight Channel-Dilation-Concatenation Net

“…This global attention mechanism can significantly increase the classification accuracy of the network. Fan et al [12] introduced a multi-scale learning and attention enhancement network to range data fusion classification in an end-to-end manner,simplifying the network structure and making network training more efficient.. Zhang et al [13] proposed a multimodal attention-aware CNN which used an attention mechanism to enhance the classification performance of light detection and ranging (LiDAR) data. Tu et al [14] designed a global-local hierarchical weighted fusion architecture to do hyperspectral image classification, effectively integrating spectral and spatial features to improve classification accuracy.…”

Contribution of Geometric Feature Analysis for Deep Learning Classification Algorithms of Urban LiDAR Data

“…These strategies addressed the issue of local-feature clustering by leveraging semantic similarity in the local region and incorporating global features in the local feature aggregation. Fan et al [26] utilized the Multiscale Learning and Attention Enhancement Network (MSLAENet) for the end-to-end classification of HSI and LiDAR data. The network employed a two-branch CNN structure with self-calibrated convolutions and a hierarchical residual structure, enabling the extraction of spectral and spatial information at multiple scales.…”