Embedding Earth: Self-supervised contrastive pre-training for dense land cover classification

Abstract

In training machine learning models for land cover semantic segmentation there is a stark contrast between the availability of satellite imagery to be used as inputs and ground truth data to enable supervised learning. While thousands of new satellite images become freely available on a daily basis, getting ground truth data is still very challenging, time consuming and costly. In this paper we present Embedding Earth a selfsupervised contrastive pre-training method for leveraging the large availability of satellite imagery to improve performance on downstream dense land cover classification tasks. Performing an extensive experimental evaluation spanning four countries and two continents we use models pre-trained with our proposed method as initialization points for supervised land cover semantic segmentation and observe significant improvements up to 25% absolute mIoU. In every case tested we outperform random initialization, especially so when ground truth data are scarse. Through a series of ablation studies we explore the qualities of the proposed approach and find that learnt features can generalize between disparate regions opening up the possibility of using the proposed pre-training scheme as a replacement to random initialization for Earth observation tasks. Code will be uploaded soon at https://github.com/michaeltrs/DeepSatModels.

Semantic-aware Dense Representation Learning for Remote Sensing Image Change Detection

“…Very recently, many contrastive methods have been applied in RS to obtain in-domain pre-trained models that benefit downstream tasks, including land-cover classification [22, 24-28, 60, 61], semantic segmentation [29][30][31][32], and change detection [16][17][18]. Most existing methods apply InfoNCE loss [18, 22, 24-28, 30, 33, 62] or triple loss [17,60] on the constructed positive and negative pairs.…”

“…Contrastive SSL [20,21] could learn useful representations from massive unlabeled data by pulling together representations of semantically similar samples (i.e., positive pairs) and pushing away those of dissimilar samples (i.e., negative pairs). Very recently, contrastive methods have been introduced in the RS domain [16][17][18][22][23][24][25][26][27][28][29][30][31][32][33] and have shown promising performance for the downstream supervised CD task [16][17][18].…”

Cropformer: A new generalized deep learning classification approach for multi-scenario crop classification

“…Pre-training as an effective training method has been applied to land use classification to accelerate the convergence of the training process (Zhao et al, 2017). The self-supervised pre-training approach can improve the utilization of labeled samples in land cover classification (Tarasiou and Zafeiriou, 2022). Unlabeled data as pre-training data can effectively improve crop classification accuracy and reduce the use of labeled samples (Yuan and Lin, 2021;Yuan et al, 2022).…”

GFCNet: Contrastive Learning Network with Geography Feature Space Joint Negative Sample Correction for Land Cover Classification

“…In recent years, with the continuous improvement in the volume and spatial resolution of remote sensing images covering the world [15], compared with supervised learning models that use high-cost labeled data as supervised signals, self-supervised contrastive learning models driven by a large number of unlabeled data are expected to become an effective solution for large-scale land cover classification with limited labeled data [16][17][18][19][20].…”

“…These methods effectively alleviate the class confusion, but they only use the feature space relationship of the image, and do not use the geospatial relationship of the image, although the geospatial relationship is easy to obtain for remote sensing images. The second challenge is that the existing self-supervised contrastive learning models simply use the single-scale features extracted by the feature extractor for land cover classification tasks [16][17][18][19], which limits the ability of the model to capture different scales of ground objects. To address this challenge, the existing selfsupervised contrastive learning models consider adding local contrastive modules [28] or dense contrastive modules [29,42] in the pretraining stage of feature extraction.…”