Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding

Stojnić, Vladan; Risojević, Vladimir

doi:10.1109/cvprw53098.2021.00129

Cited by 83 publications

(60 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most works explore natural remote sensing imagery [10], [12], [39], while some works explore abstractions of remote sensing imagery within the context of machine learning [28], [35], [43]. Methods in these areas almost exclusively follow supervised or unsupervised approaches, with very few, such as [34], following self-supervised approaches. [31] and [34] are of particular interest as they show that self-supervised pretraining (using Momentum Contrast [8] and Contrastive Multiview Coding [36] respectively) can outperform supervised pretraining in the remote sensing imagery domain, which forms the inspiration for most exploration in this paper.…”

Section: Related Workmentioning

confidence: 99%

“…Methods in these areas almost exclusively follow supervised or unsupervised approaches, with very few, such as [34], following self-supervised approaches. [31] and [34] are of particular interest as they show that self-supervised pretraining (using Momentum Contrast [8] and Contrastive Multiview Coding [36] respectively) can outperform supervised pretraining in the remote sensing imagery domain, which forms the inspiration for most exploration in this paper. In this work, we explore established tasks from remote sensing based urban computing using machine learning techniques leveraging self-supervision, which falls under the broader area of deep learning.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Self-Supervision, Remote Sensing and Abstraction: Representation Learning Across 3 Million Locations

Seneviratne,

Nice,

Wijnands

et al. 2022

Preprint

View full text Add to dashboard Cite

Self-supervision based deep learning classification approaches have received considerable attention in academic literature. However, the performance of such methods on remote sensing imagery domains remains under-explored. In this work, we explore contrastive representation learning methods on the task of imagery-based city classification, an important problem in urban computing. We use satellite and map imagery across 2 domains, 3 million locations and more than 1500 cities. We show that self-supervised methods can build a generalizable representation from as few as 200 cities, with representations achieving over 95% accuracy in unseen cities with minimal additional training. We also find that the performance discrepancy of such methods, when compared to supervised methods, induced by the domain discrepancy between natural imagery and abstract imagery is significant for remote sensing imagery. We compare all analysis against existing supervised models from academic literature and open-source our models 1 for broader usage and further criticism.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Self-Supervision, Remote Sensing and Abstraction: Representation Learning Across 3 Million Locations

Seneviratne,

Nice,

Wijnands

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Unlike in the field of classic computer vision, self-supervised learning of aerial images has not yet been fully studied. Stojnic et al [32] apply Contrastive Multiview Coding [35] to learn aerial image representations on both RGB and multispectral remote sensing images. [17] proposes a method based on contrastive learning with different image augmentations.…”

Section: Related Workmentioning

confidence: 99%

Self-Supervised Pretraining and Controlled Augmentation Improve Rare Wildlife Recognition in UAV Images

Zheng¹,

Kellenberger²,

Gong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Automated animal censuses with aerial imagery are a vital ingredient towards wildlife conservation. Recent models are generally based on deep learning and thus require vast amounts of training data. Due to their scarcity and minuscule size, annotating animals in aerial imagery is a highly tedious process. In this project, we present a methodology to reduce the amount of required training data by resorting to self-supervised pretraining. In detail, we examine a combination of recent contrastive learning methodologies like Momentum Contrast (MoCo) and Cross-Level Instance-Group Discrimination (CLD) to condition our model on the aerial images without the requirement for labels. We show that a combination of MoCo, CLD, and geometric augmentations outperforms conventional models pretrained on Im-ageNet by a large margin. Crucially, our method still yields favorable results even if we reduce the number of training animals to just 10%, at which point our best model scores double the recall of the baseline at similar precision. This effectively allows reducing the number of required annotations to a fraction while still being able to train highaccuracy models in such highly challenging settings.

show abstract

“…Contrastive SSL [20,21] could learn useful representations from massive unlabeled data by pulling together representations of semantically similar samples (i.e., positive pairs) and pushing away those of dissimilar samples (i.e., negative pairs). Very recently, contrastive methods have been introduced in the RS domain [16][17][18][22][23][24][25][26][27][28][29][30][31][32][33] and have shown promising performance for the downstream supervised CD task [16][17][18].…”

Section: Introductionmentioning

confidence: 99%

Semantic-aware Dense Representation Learning for Remote Sensing Image Change Detection

Chen,

Li,

Chen

et al. 2022

Preprint

View full text Add to dashboard Cite

Training deep learning-based change detection (CD) model heavily depends on labeled data. Contemporary transfer learning-based methods to alleviate the CD label insufficiency mainly upon ImageNet pre-training. A recent trend is using remote sensing (RS) data to obtain in-domain representations via supervised or self-supervised learning (SSL). Here, different from traditional supervised pre-training that learns the mapping from image to label, we leverage semantic supervision in a contrastive manner. There are typically multiple objects of interest (e.g., buildings) distributed in varying locations in RS images. We propose dense semantic-aware pre-training for RS image CD via sampling multiple class-balanced points. Instead of manipulating image-level representations that lack spatial information, we constrain pixel-level cross-view consistency and cross-semantic discrimination to learn spatially-sensitive features, thus benefiting downstream dense CD. Apart from learning illumination invariant features, we fulfill consistent foreground features insensitive to irrelevant background changes via a synthetic view using background swapping. We additionally achieve discriminative representations to distinguish foreground land-covers and other backgrounds. We collect large-scale image-mask pairs freely available in the RS community for pre-training. Extensive experiments on three CD datasets verify the effectiveness of our method. Ours significantly outperforms ImageNet, in-domain supervision, and several SSL methods. Empirical results indicate ours well alleviates data insufficiency in CD. Notably, we achieve competitive results using only 20% training data than random baseline using 100% data. Both quantitative and qualitative results demonstrate the generalization ability of our pre-trained model to downstream images even remaining domain gaps with the pre-training data. Our data and code will make public.

show abstract

Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding

Cited by 83 publications

References 23 publications

Self-Supervision, Remote Sensing and Abstraction: Representation Learning Across 3 Million Locations

Self-Supervision, Remote Sensing and Abstraction: Representation Learning Across 3 Million Locations

Self-Supervised Pretraining and Controlled Augmentation Improve Rare Wildlife Recognition in UAV Images

Semantic-aware Dense Representation Learning for Remote Sensing Image Change Detection

Contact Info

Product

Resources

About