Multi-View Urban Scene Classification with a Complementary-Information Learning Model

Geng, Wanxuan; Zhou, Weixun; Jin, Shuanggen

doi:10.14358/pers.21-00062r2

Cited by 7 publications

(10 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As mentioned in Section 1, existing multi-view fusion methods can be roughly classified as data-level, feature-level, and decision-level. In this experiment, one data-level fusion method (six-channel [42]), two feature-level fusion methods (feature concatenation [31] and CILM [43]), and five decision-level fusion methods (maximum [31], minimum [31], sum [31], product [31] and SFWS [44]) were chosen to compare with the proposed evidential fusion. These methods are briefly described below.…”

Section: Viewsmentioning

confidence: 99%

“…Feature concatenation [31]: A Siamese-like CNN is used to concatenate the intermediate feature tensors before the first convolution layer that doubles its amount of kernels. • CILM [43]: The loss function of contrast learning is combined with CE Loss in this method, allowing the features extracted by the two subnetworks to be fused without sharing any weight. • Maximum [31]: Each view employs an independent DNN to obtain its prediction result, which consists of a class label and its probability.…”

mentioning

confidence: 99%

“…Concat [31]. CILM[43] Dec.-Lev [31,44]. Proposed AiRound 9.16 ± 0.01 9.21 ± 0.02 9.38 ± 0.02 9.37 ± 0.01 9.42 ± 0.01 Cv-BrCT 8.01 ± 0.01 8.06 ± 0.02 8.20 ± 0.02 8.19 ± 0.01 8.23 ± 0.01…”

mentioning

confidence: 99%

See 2 more Smart Citations

Credible Remote Sensing Scene Classification Using Evidential Fusion on Aerial-Ground Dual-View Images

et al. 2023

View full text Add to dashboard Cite

Due to their ability to offer more comprehensive information than data from a single view, multi-view (e.g., multi-source, multi-modal, multi-perspective) data are being used more frequently in remote sensing tasks. However, as the number of views grows, the issue of data quality is becoming more apparent, limiting the potential benefits of multi-view data. Although recent deep neural network (DNN)-based models can learn the weight of data adaptively, a lack of research on explicitly quantifying the data quality of each view when fusing them renders these models inexplicable, performing unsatisfactorily and inflexibly in downstream remote sensing tasks. To fill this gap, in this paper, evidential deep learning is introduced to the task of aerial-ground dual-view remote sensing scene classification to model the credibility of each view. Specifically, the theory of evidence is used to calculate an uncertainty value which describes the decision-making risk of each view. Based on this uncertainty, a novel decision-level fusion strategy is proposed to ensure that the view with lower risk obtains more weight, making the classification more credible. On two well-known, publicly available datasets of aerial-ground dual-view remote sensing images, the proposed approach achieves state-of-the-art results, demonstrating its effectiveness.

show abstract

Section: Viewsmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Credible Remote Sensing Scene Classification Using Evidential Fusion on Aerial-Ground Dual-View Images

et al. 2023

View full text Add to dashboard Cite

show abstract

“…In other studies [24,60], inpainting and relative position pretext tasks are used for segmentation and classification. In recent years, the contrastive-learning-based SSL approach has also been widely used in remote sensing [27,[61][62][63]. Tile2Vec was the first self-supervised approach using contrastive learning for remote sensing image representation learning [64].…”

Section: Contrastive Learning Basedmentioning

confidence: 99%

A Generic Self-Supervised Learning (SSL) Framework for Representation Learning from Spectral–Spatial Features of Unlabeled Remote Sensing Imagery

Zhang,

Han

2023

Remote Sensing

View full text Add to dashboard Cite

Remote sensing data has been widely used for various Earth Observation (EO) missions such as land use and cover classification, weather forecasting, agricultural management, and environmental monitoring. Most existing remote-sensing-data-based models are based on supervised learning that requires large and representative human-labeled data for model training, which is costly and time-consuming. The recent introduction of self-supervised learning (SSL) enables models to learn a representation from orders of magnitude more unlabeled data. The success of SSL is heavily dependent on a pre-designed pretext task, which introduces an inductive bias into the model from a large amount of unlabeled data. Since remote sensing imagery has rich spectral information beyond the standard RGB color space, it may not be straightforward to extend to the multi/hyperspectral domain the pretext tasks established in computer vision based on RGB images. To address this challenge, this work proposed a generic self-supervised learning framework based on remote sensing data at both the object and pixel levels. The method contains two novel pretext tasks, one for object-based and one for pixel-based remote sensing data analysis methods. One pretext task is used to reconstruct the spectral profile from the masked data, which can be used to extract a representation of pixel information and improve the performance of downstream tasks associated with pixel-based analysis. The second pretext task is used to identify objects from multiple views of the same object in multispectral data, which can be used to extract a representation and improve the performance of downstream tasks associated with object-based analysis. The results of two typical downstream task evaluation exercises (a multilabel land cover classification task on Sentinel-2 multispectral datasets and a ground soil parameter retrieval task on hyperspectral datasets) demonstrate that the proposed SSL method learns a target representation that covers both spatial and spectral information from massive unlabeled data. A comparison with currently available SSL methods shows that the proposed method, which emphasizes both spectral and spatial features, outperforms existing SSL methods on multi- and hyperspectral remote sensing datasets. We believe that this approach has the potential to be effective in a wider range of remote sensing applications and we will explore its utility in more remote sensing applications in the future.

show abstract

“…Remote sensing. While contrastive self-supervised learning is relatively new, the wide use of contrastive loss in remote sensing [178,179,180,181,182,183] can date back to [184], where the authors imposed a supervised contrastive regularization term on the CNN features for remote sensing scene classification. The first self-supervised work making use of contrastive learning for remote sensing image representation learning is Tile2Vec proposed by Jean et al [177].…”

Section: Infoncementioning

confidence: 99%

Self-Supervised Learning in Remote Sensing: A review

Wang

Albrecht

Braham

et al. 2022

IEEE Geosci. Remote Sens. Mag.

136

View full text Add to dashboard Cite

In deep learning research, self-supervised learning (SSL) has received great attention triggering interest within both the computer vision and remote sensing communities. While there has been a big success in computer vision, most of the potential of SSL in the domain of earth observation remains locked. In this paper, we provide an introduction to, and a review of the concepts and latest developments in SSL for computer vision in the context of remote sensing. Further, we provide a preliminary benchmark of modern SSL algorithms on popular remote sensing datasets, verifying the potential of SSL in remote sensing and providing an extended study on data augmentations. Finally, we identify a list of promising directions of future research in SSL for earth observation (SSL4EO) to pave the way for fruitful interaction of both domains.

show abstract

Multi-View Urban Scene Classification with a Complementary-Information Learning Model

Cited by 7 publications

References 21 publications

Credible Remote Sensing Scene Classification Using Evidential Fusion on Aerial-Ground Dual-View Images

Credible Remote Sensing Scene Classification Using Evidential Fusion on Aerial-Ground Dual-View Images

A Generic Self-Supervised Learning (SSL) Framework for Representation Learning from Spectral–Spatial Features of Unlabeled Remote Sensing Imagery

Self-Supervised Learning in Remote Sensing: A review

Contact Info

Product

Resources

About