Classification of Hyperspectral-LiDAR Dual-View Data Using Hybrid Feature and Trusted Decision Fusion

Abstract

As remote sensing technology continues to evolve, the integration of multi-view data, including HSI and LiDAR data, has emerged as a pivotal research area aimed at enhancing the precision of classification. However, most current multimodal data fusion methods follow a process of first extracting features from each modality, then combining these features using a fusion strategy, and finally performing classification. This approach may result in the diminution of original information during the feature fusion process and relies heavily on the performance of the Softmax function during classification, without adequately considering the trustworthiness of the results. To address the above issues, this paper presented a hybrid feature and trusted decision fusion (HFTDF) method for dual-view remote sensing data classification. In terms of the research method, the approach first performs preliminary feature extraction on dual-view data using shallow CNN models, while implementing a shallow fusion strategy to integrate original information from different data sources at an early stage. Next, it leverages the proficiency of CNNs in learning localized characteristics and the potential of the Transformer in terms of its handling of overarching information, conducting hybrid feature learning on data from each view. Additionally, a deep fusion strategy serves to investigate the intricate interrelations among diverse perspectives. Finally, evidence theory is applied to model the uncertainty of classification results, generating trusted vectors, and a trusted decision fusion strategy is employed to merge the trusted information from each modality at the decision level, thereby enhancing the reliability of the results. HFTDF achieves overall classification accuracies of 94.68%, 99.17%, and 82.05% on the Houston 2013, Trento, and MUUFL datasets, respectively, when only 20 samples of each class are used for training. The classification results of the experiments reveal that HFTDF outperforms in the classification of dual-view data.