Deep Hierarchical Vision Transformer for Hyperspectral and LiDAR Data Classification

Xue, Zhixiang; Tan, Xiong; Yu, Xi; Liu, Bing; Yu, Anzhu; Zhang, Pengqiang

doi:10.1109/tip.2022.3162964

Cited by 106 publications

(17 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Xue et al in [37] proposed deep hierarchical ViT (DHViT) for the hyperspectral and light detection and ranging (LiDAR) data classification. The authors used the Trento, Houston 2013, and Houston 2018 datasets and obtained accuracies of 99.58%, 99.55%, and 96.40%, respectively.…”

Section: Vits For Image Classificationmentioning

confidence: 99%

A Comprehensive Survey of Transformers for Computer Vision

Jamil

Piran

Kwon

2023

Drones

View full text Add to dashboard Cite

As a special type of transformer, vision transformers (ViTs) can be used for various computer vision (CV) applications. Convolutional neural networks (CNNs) have several potential problems that can be resolved with ViTs. For image coding tasks such as compression, super-resolution, segmentation, and denoising, different variants of ViTs are used. In our survey, we determined the many CV applications to which ViTs are applicable. CV applications reviewed included image classification, object detection, image segmentation, image compression, image super-resolution, image denoising, anomaly detection, and drone imagery. We reviewed the state of the-art and compiled a list of available models and discussed the pros and cons of each model.

show abstract

Section: Vits For Image Classificationmentioning

confidence: 99%

A Comprehensive Survey of Transformers for Computer Vision

Jamil

Piran

Kwon

2023

Drones

View full text Add to dashboard Cite

show abstract

“…The crux of multi-source data lies in devising strategies to address the collaborative processing of multi-source features, enabling harmonious representation across disparate datasets. Xue Z et al proposed the network architecture DHViT [13], which incorporates hyperspectral images and Lidar data, and introduced a cross-attention feature fusion module to enhance collaborative classification performance. Additionally, MIFNet [14] presented a global dependence fusion module (GDFM) for feature fusion across diverse data sources.…”

Section: Introductionmentioning

confidence: 99%

Multisource remote sensing image classification based on adaptive pooling transformer and multilevel correction

Wang,

Shi,

Gao

et al. 2024

Fifteenth International Conference on Graphics and Image Processing (ICGIP 2023)

View full text Add to dashboard Cite

Deep learning and neural networks have ascended to prominence as the prevailing methodologies for the computational manipulation of remote sensing imagery. The collection and utilization of single remote sensing data inevitably encounter limitations, such as the arduous task of subdividing akin characteristics within hyperspectral data. Endeavors to collaborate with multiple remote sensing data sources are intensifying, aiming to attain more refined and comprehensive observational outcomes. In this article, we propose a comprehensive system framework for the processing of multi-source remote sensing image classification, named the adaptive pooling transformer network (APTnet). Primarily, Convolutional Neural Networks (CNNs) are harnessed to extract distinctive attributes from both hyperspectral images (HSI) and synthetic aperture radar images (SAR). Furthermore, we propose a technique for the fusion of remote sensing data, known as the cross-modal correction fusion method (CMCF), enabling interactive learning and rectification of multi-source features across diverse strata. Additionally, an adaptive pooling Transformer method (APT) is put forth to fortify the existing attributes and amplify their capacity for expression. The experiments conducted on the Berlin and Augsburg datasets exhibit commendable performance, complemented by exhaustive comparative analyses with prevailing methodologies.

show abstract

“…Subsequently, to further exploit high-level semantic features, convolutional neural networks (CNNs) are employed for multisource data classification [13]. Encoder-decoder network [14], coupled CNNs [15], Gabor CNN [16], cross attention [17], and Transformer [18] are used to extract representative multisource features, and these methods have achieved promising performance.…”

Section: Introductionmentioning

confidence: 99%

Nearest Neighbor-Based Contrastive Learning for Hyperspectral and LiDAR Data Classification

Wang¹,

Gao²,

Dong³

et al. 2023

Preprint

View full text Add to dashboard Cite

The joint hyperspectral image (HSI) and LiDAR data classification aims to interpret ground objects at more detailed and precise level. Although deep learning methods have shown remarkable success in the multisource data classification task, self-supervised learning has rarely been explored. It is commonly nontrivial to build a robust self-supervised learning model for multisource data classification, due to the fact that the semantic similarities of neighborhood regions are not exploited in existing contrastive learning framework. Furthermore, the heterogeneous gap induced by the inconsistent distribution of multisource data impedes the classification performance. To overcome these disadvantages, we propose a Nearest Neighborbased Contrastive Learning Network (NNCNet), which takes full advantage of large amounts of unlabeled data to learn discriminative feature representations. Specifically, we propose a nearest neighbor-based data augmentation scheme to use enhanced semantic relationships among nearby regions. The intermodal semantic alignments can be captured more accurately. In addition, we design a bilinear attention module to exploit the second-order and even high-order feature interactions between the HSI and LiDAR data. Extensive experiments on four public datasets demonstrate the superiority of our NNCNet over state-of-the-art methods. The source codes are available at https://github.com/summitgao/NNCNet.

show abstract

Deep Hierarchical Vision Transformer for Hyperspectral and LiDAR Data Classification

Cited by 106 publications

References 45 publications

A Comprehensive Survey of Transformers for Computer Vision

A Comprehensive Survey of Transformers for Computer Vision

Multisource remote sensing image classification based on adaptive pooling transformer and multilevel correction

Nearest Neighbor-Based Contrastive Learning for Hyperspectral and LiDAR Data Classification

Contact Info

Product

Resources

About