Fusion of Hyperspectral and Lidar Data Based On Dual-Branch Convolutional Neural Network

Wang, Jinzhe; Zhang, Junping; Guo, Qingle; Li, Tong

doi:10.1109/igarss.2019.8899332

Cited by 11 publications

(2 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the HSI feature extraction branch of these methods often use 2D-CNNs, which can only capture spatial features of HSI and ignore the spectral information from the original data, leading to poor feature fusion quality. Although 3D-CNN was also employed to extract spectral features from HSI [25][26][27], the classification results are still unsatisfactory. This is because the traditional 3D-CNNs can only extract features within a fixed-size range around the same pixel, limiting its feature extraction capability and hindering the classification performance [28].…”

Section: Introductionmentioning

confidence: 99%

Attention-Guided Fusion and Classification for Hyperspectral and LiDAR Data

Huang,

Zhang,

Yang

et al. 2023

Remote Sensing

View full text Add to dashboard Cite

The joint use of hyperspectral image (HSI) and Light Detection And Ranging (LiDAR) data has been widely applied for land cover classification because it can comprehensively represent the urban structures and land material properties. However, existing methods fail to combine the different image information effectively, which limits the semantic relevance of different data sources. To solve this problem, in this paper, an Attention-guided Fusion and Classification framework based on Convolutional Neural Network (AFC-CNN) is proposed to classify the land cover based on the joint use of HSI and LiDAR data. In the feature extraction module, AFC-CNN employs the three dimensional convolutional neural network (3D-CNN) combined with a multi-scale structure to extract the spatial-spectral features of HSI, and uses a 2D-CNN to extract the spatial features from LiDAR data. Simultaneously, the spectral attention mechanism is adopted to assign weights to the spectral channels, and the cross attention mechanism is introduced to impart significant spatial weights from LiDAR to HSI, which enhance the interaction between HSI and LiDAR data and leverage the fusion information. Then two feature branches are concatenated and transferred to the feature fusion module for higher-level feature extraction and fusion. In the fusion module, AFC-CNN adopts the depth separable convolution connected through the residual structures to obtain the advanced features, which can help reduce computational complexity and improve the fitting ability of the model. Finally, the fused features are sent into the linear classification module for final classification. Experimental results on three datasets, i.e., Houston, MUUFL and Trento datasets show that the proposed AFC-CNN framework achieves better classification accuracy compared with the state-of-the-art algorithms. The overall accuracy of AFC-CNN on Houston, MUUFL and Trento datasets are 94.2%, 95.3% and 99.5%, respectively.

show abstract

Section: Introductionmentioning

confidence: 99%

Attention-Guided Fusion and Classification for Hyperspectral and LiDAR Data

Huang,

Zhang,

Yang

et al. 2023

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Xu [26] et al designed a CNN with cascaded blocks, Hang [27] et al proposed a coupled CNN network to reduce model complexity and improve classification performance through weight sharing. Zhang [28] et al designed an unsupervised feature extraction framework based on CNN, some scholars also introduced 3DCNN in the HSI branch to better extract the spatial spectral information of HSI [29]. Different from the CNN approach, Hong [30] et al built a deep network based on autoencoder for classification of hyperspectral and LiDAR data.…”

mentioning

confidence: 99%

MSLAENet: Multiscale Learning and Attention Enhancement Network for Fusion Classification of Hyperspectral and LiDAR Data

Fan

Qian

Qin

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

The effective use of multimodal data to obtain accurate land cover information has become an interesting and challenging research topic in the field of remote sensing. In this paper, we propose a new method, multi-scale learning and attention enhancement network (MSLAENet), to implement Hyperspectral image (HSI) and light detection and ranging (LiDAR) data fusion classification in an end-to-end manner. Specifically, our model consists of three main modules. First, we design the composite attention (CA) module, which adopts self-attention to enhance the feature representations of HSI and LiDAR data, respectively, and cross-attention to achieve crossmodal information enhancement. Second, the proposed multiscale learning (MSL) module combines self-calibrated convolutions and hierarchical residual structure to extract different scales of information to further improve the representation capability of the model. Finally, the attention-based feature fusion (FF) module fully considers the complementary information properties between different modalities and adaptively fuses heterogeneous features from different modalities. To test the performance of MSLAENet, we conduct experiments on three multimodal remote sensing datasets and compare them with the state-ofthe-art fusion model, which demonstrates the effectiveness and superiority of the model.

show abstract

Hyperspectral and LiDAR data fusion in features based classification

Abdulrahman

2021

Arab J Geosci

View full text Add to dashboard Cite

Fusion of Hyperspectral and Lidar Data Based On Dual-Branch Convolutional Neural Network

Cited by 11 publications

References 5 publications

Attention-Guided Fusion and Classification for Hyperspectral and LiDAR Data

Attention-Guided Fusion and Classification for Hyperspectral and LiDAR Data

MSLAENet: Multiscale Learning and Attention Enhancement Network for Fusion Classification of Hyperspectral and LiDAR Data

Hyperspectral and LiDAR data fusion in features based classification

Contact Info

Product

Resources

About