Evaluation of Split-Brain Autoencoders for High-Resolution Remote Sensing Scene Classification

Stojnić, Vladan; Risojević, Vladimir

doi:10.23919/elmar.2018.8534634

Cited by 9 publications

(10 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One limitation of this architecture compared to other asymmetric Siamese methods is the need to use two different encoders, even during inference, since each one is trained on different data channels than the other. Based on their experiments on the Resisc-45 and AID datasets using the RGB and LAB color spaces, the authors show that the method could yield competitive results even with few unlabeled training images [54]. They expect a high potential of split-brain AEs on multi-spectral remote sensing images in perspective works.…”

Section: Generativementioning

confidence: 98%

See 1 more Smart Citation

Self-Supervised Learning for Scene Classification in Remote Sensing: Current State of the Art and Perspectives

2022

View full text Add to dashboard Cite

Deep learning methods have become an integral part of computer vision and machine learning research by providing significant improvement performed in many tasks such as classification, regression, and detection. These gains have been also observed in the field of remote sensing for Earth observation where most of the state-of-the-art results are now achieved by deep neural networks. However, one downside of these methods is the need for large amounts of annotated data, requiring lots of labor-intensive and expensive human efforts, in particular for specific domains that require expert knowledge such as medical imaging or remote sensing. In order to limit the requirement on data annotations, several self-supervised representation learning methods have been proposed to learn unsupervised image representations that can consequently serve for downstream tasks such as image classification, object detection or semantic segmentation. As a result, self-supervised learning approaches have been considerably adopted in the remote sensing domain within the last few years. In this article, we review the underlying principles developed by various self-supervised methods with a focus on scene classification task. We highlight the main contributions and analyze the experiments, as well as summarize the key conclusions, from each study. We then conduct extensive experiments on two public scene classification datasets to benchmark and evaluate different self-supervised models. Based on comparative results, we investigate the impact of individual augmentations when applied to remote sensing data as well as the use of self-supervised pre-training to boost the classification performance with limited number of labeled samples. We finally underline the current trends and challenges, as well as perspectives of self-supervised scene classification.

show abstract

Section: Generativementioning

confidence: 98%

“…Another early use of generative models in SSL applied to remote sensing is proposed by [54], where the authors evaluate the use of a split-brain autoencoder for self-supervised image representation. During the process of learning to reconstruct the input image, autoencoders discover relevant information about the data distribution.…”

Section: Generativementioning

confidence: 99%

Self-Supervised Learning for Scene Classification in Remote Sensing: Current State of the Art and Perspectives

2022

View full text Add to dashboard Cite

show abstract

“…Again, they do not address the few shot problem in their work. In 34 , Stojnic et al study the Contrastive Multiview Coding (CMC) method 35 for self-supervised pre-training application of SSL. They analyze the influence of the number and domain of images used for self-supervised pre-training on the performance on downstream tasks.…”

Section: Related Workmentioning

confidence: 99%

Self-supervised learning for remote sensing scene classification under the few shot scenario

Alosaimi

Alhichri

Bazi

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Scene classification is a crucial research problem in remote sensing (RS) that has attracted many researchers recently. It has many challenges due to multiple issues, such as: the complexity of remote sensing scenes, the classes overlapping (as a scene may contain objects that belong to foreign classes), and the difficulty of gaining sufficient labeled scenes. Deep learning (DL) solutions and in particular convolutional neural networks (CNN) are now state-of-the-art solution in RS scene classification; however, CNN models need huge amounts of annotated data, which can be costly and time-consuming. On the other hand, it is relatively easy to acquire large amounts of unlabeled images. Recently, Self-Supervised Learning (SSL) is proposed as a method that can learn from unlabeled images, potentially reducing the need for labeling. In this work, we propose a deep SSL method, called RS-FewShotSSL, for RS scene classification under the few shot scenario when we only have a few (less than 20) labeled scenes per class. Under this scenario, typical DL solutions that fine-tune CNN models, pre-trained on the ImageNet dataset, fail dramatically. In the SSL paradigm, a DL model is pre-trained from scratch during the pretext task using the large amounts of unlabeled scenes. Then, during the main or the so-called downstream task, the model is fine-tuned on the labeled scenes. Our proposed RS-FewShotSSL solution is composed of an online network and a target network both using the EfficientNet-B3 CNN model as a feature encoder backbone. During the pretext task, RS-FewShotSSL learns discriminative features from the unlabeled images using cross-view contrastive learning. Different views are generated from each image using geometric transformations and passed to the online and target networks. Then, the whole model is optimized by minimizing the cross-view distance between the online and target networks. To address the problem of limited computation resources available to us, our proposed method uses a novel DL architecture that can be trained using both high-resolution and low-resolution images. During the pretext task, RS-FewShotSSL is trained using low-resolution images, thereby, allowing for larger batch sizes which significantly boosts the performance of the proposed pipeline on the task of RS classification. In the downstream task, the target network is discarded, and the online network is fine-tuned using the few labeled shots or scenes. Here, we use smaller batches of both high-resolution and low-resolution images. This architecture allows RS-FewshotSSL to benefit from both large batch sizes and full image sizes, thereby learning from the large amounts of unlabeled data in an effective way. We tested RS-FewShotSSL on three RS public datasets, and it demonstrated a significant improvement compared to other state-of-the-art methods such as: SimCLR, MoCo, BYOL and IDSSL.

show abstract

“…Lately, some research has been done in the area of applying self-supervised learning to the analysis of remote sensing images. In [31,30] the authors analyzed the possibilities of using split-brain autoencoder [44] in the analysis of aerial images. They analyzed the influence of the number of images used for self-supervised learning, as well as usage of different color channels, on the results obtained on the downstream task of aerial image classification.…”

Section: Self-supervised Learning In Remote Sensingmentioning

confidence: 99%

“…However, the application of self-supervised learning methods in remote sensing has not been studied a lot. Most of these applications either used small amounts, up to 50,000 images, of unlabeled training data [31,30,35] or tested learned representations on a small number of datasets [1,19,38].…”

Section: Introductionmentioning

confidence: 99%

Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding

Stojnić¹,

Risojević²

2021

Preprint

View full text Add to dashboard Cite

In recent years self-supervised learning has emerged as a promising candidate for unsupervised representation learning. In the visual domain its applications are mostly studied in the context of images of natural scenes. However, its applicability is especially interesting in specific areas, like remote sensing and medicine, where it is hard to obtain huge amounts of labeled data. In this work, we conduct an extensive analysis of the applicability of self-supervised learning in remote sensing image classification. We analyze the influence of the number and domain of images used for self-supervised pre-training on the performance on downstream tasks. We show that, for the downstream task of remote sensing image classification, using self-supervised pre-training on remote sensing images can give better results than using supervised pre-training on images of natural scenes. Besides, we also show that self-supervised pretraining can be easily extended to multispectral images producing even better results on our downstream tasks.

show abstract

Evaluation of Split-Brain Autoencoders for High-Resolution Remote Sensing Scene Classification

Cited by 9 publications

References 4 publications

Self-Supervised Learning for Scene Classification in Remote Sensing: Current State of the Art and Perspectives

Self-Supervised Learning for Scene Classification in Remote Sensing: Current State of the Art and Perspectives

Self-supervised learning for remote sensing scene classification under the few shot scenario

Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding

Contact Info

Product

Resources

About