Accelerating Ecological Sciences from Above: Spatial Contrastive Learning for Remote Sensing

Abstract

The rise of neural networks has opened the door for automatic analysis of remote sensing data. A challenge to using this machinery for computational sustainability is the necessity of massive labeled data sets, which can be cost-prohibitive for many non-profit organizations. The primary motivation for this work is one such problem; the efficient management of invasive species -- invading flora and fauna that are estimated to cause damages in the billions of dollars annually. As an ongoing collaboration with the New York Natural Heritage Program, we consider the use of unsupervised deep learning techniques for dimensionality reduction of remote sensing images, which can reduce sample complexity for downstream tasks and decreases the need for large labeled data sets. We consider spatially augmenting contrastive learning by training neural networks to correctly classify two nearby patches of a landscape as such. We demonstrate that this approach improves upon previous methods and naive classification for a large-scale data set of remote sensing images derived from invasive species observations obtained over 30 years. Additionally, we simulate deployment in the field via active learning and evaluate this method on another important challenge in computational sustainability -- landcover classification -- and again find that it outperforms previous baselines.

AutoLTS: Automating Cycling Stress Assessment via Contrastive Learning and Spatial Post-processing

“…Contrastive learning, which learns data representations by contrasting similar and dissimilar data samples, has received growing attention in computer vision. Such techniques usually leverage a contrastive loss to guide the data encoder to pull together similar samples in an embedding space, which has been shown to facilitate downstream learning in many applications (Zhao et al 2021;Bengar et al 2021;Bjorck et al 2021), especially when data labels are unavailable or scarce. To date, most contrastive learning approaches are designed in unsupervised settings (Gutmann and Hyvärinen 2010;Sohn 2016;Oord, Li, and Vinyals 2018;Hjelm et al 2018;Wu et al 2018;Bachman, Hjelm, and Buchwalter 2019;He et al 2020;Chen et al 2020).…”

Self-supervised Pretraining of Vision Transformers for Earth Observation

“…In designing contrastive pretext tasks, remote sensing researchers built positive pairs by leveraging spatial [80,81,82,83,84,85,86,87,88,89,90], temporal [91,92,93],…”