Semi-Supervised Multitask Learning on Multispectral Satellite Images Using Wasserstein Generative Adversarial Networks (GANs) for Predicting Poverty

Perez, Anthony; Ganguli, Swetava; Ermon, Stefano; Azzari, George; Burke, Marshall; Lobell, David B.

doi:10.48550/arxiv.1902.11110

Cited by 8 publications

(8 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Applying Deep Learning to Remote Sensed Imagery Multiple projects have leveraged satellite imagery to answer various questions on land use, road quality, object detection, consumption expenditure: by linking sparse ground truth with abundant imagery, researchers can extrapolate trends in existing data to areas where labeled data do not exist [35], [10], [19]. Alternatively, some works have proposed neural network architectures that sidestep training data constraints and the relative lack of labeled ground-truth in remote areas [24] [30]. Jean et al combine Google maps daytime images (provided by DigitalGlobe), nighttime lighting, and survey data to estimate poverty for multiple African countries [29].…”

Section: Related Workmentioning

confidence: 99%

Learning to segment from misaligned and partial labels

Fobi¹,

Conlon²,

Taneja³

et al. 2020

Preprint

View full text Add to dashboard Cite

To extract information at scale, researchers increasingly apply semantic segmentation techniques to remotely-sensed imagery. While fully-supervised learning enables accurate pixel-wise segmentation, compiling the exhaustive datasets required is often prohibitively expensive. As a result, many non-urban settings lack the ground-truth needed for accurate segmentation. Existing open source infrastructure data for these regions can be inexact and non-exhaustive. Open source infrastructure annotations like OpenStreetMaps are representative of this issue: while OpenStreetMaps labels provide global insights to road and building footprints, noisy and partial annotations limit the performance of segmentation algorithms that learn from them.In this paper, we present a novel and generalizable two-stage framework that enables improved pixel-wise image segmentation given misaligned and missing annotations. First, we introduce the Alignment Correction Network to rectify incorrectly registered open source labels. Next, we demonstrate a segmentation model -the Pointer Segmentation Network -that uses corrected labels to predict infrastructure footprints despite missing annotations. We test sequential performance on the Aerial Imagery for Roof Segmentation dataset, achieving a mean intersection-over-union score of 0.79; more importantly, model performance remains stable as we decrease the fraction of annotations present. We demonstrate the transferability of our method to lower quality data sources, by applying the Alignment Correction Network to OpenStreetMaps labels to correct building footprints; we also demonstrate the accuracy of the Pointer Segmentation Network in predicting cropland boundaries in California from medium resolution data. Overall, our methodology is robust for multiple applications with varied amounts of training data present, thus offering a method to extract reliable information from noisy, partial data.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning to segment from misaligned and partial labels

Fobi¹,

Conlon²,

Taneja³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…In addition, using convolutional neural networks and GANs on geospatial data in unsupervised or semi-supervised settings has also been of interest recently; especially in domains such as food security, cybersecurity, satellite tasking, etc. (Ganguli, Dunnmon, and Hau (2019); Perez et al (2019); Dunnmon et al (2019)). Conditional Adversarial Networks (Mirza and Osindero (2014); Isola et al (2016)) have been used to perform general purpose image-to-image translation.…”

Section: Related Workmentioning

confidence: 99%

GeoGAN: A Conditional GAN with Reconstruction and Style Loss to Generate Standard Layer of Maps from Satellite Images

Ganguli¹,

Garzón²,

Glaser³

2019

Preprint

Self Cite

View full text Add to dashboard Cite

Automatically generating maps from satellite images is an important task. There is a body of literature which tries to address this challenge. We created a more expansive survey of the task by experimenting with different models and adding new loss functions to improve results. We created a database of pairs of satellite images and the corresponding map of the area. Our model translates the satellite image to the corresponding standard layer map image using three main model architectures: (i) a conditional Generative Adversarial Network (GAN) which compresses the images down to a learned embedding, (ii) a generator which is trained as a normalizing flow (RealNVP) model, and (iii) a conditional GAN where the generator translates via a series of convolutions to the standard layer of a map and the discriminator input is the concatenation of the real/generated map and the satellite image. Model (iii) was by far the most promising of three models. To improve the results we also added a reconstruction loss and style transfer loss in addition to the GAN losses. The third model architecture produced the best quality of sampled images. In contrast to the other generative model where evaluation of the model is a challenging problem. since we have access to the real map for a given satellite image, we are able to assign a quantitative metric to the quality of the generated images in addition to inspecting them visually. While we are continuing to work on increasing the accuracy of the model, one challenge has been the coarse resolution of the data which upper-bounds the quality of the results of our model. Nevertheless, as will be seen in the results, the generated map is more accurate in the features it produces since the generator architecture demands a pixel-wise image translation/pixel-wise coloring. A video presentation summarizing this paper is available at: https://youtu.be/Ur0flOX-Ji0

show abstract

“…In addition, using convolutional neural networks and GANs on geospatial data in unsupervised or semi-supervised settings has also been of interest recently; especially in domains such as food security, cybersecurity, satellite tasking, etc. ((Ganguli, Garzon, and Glaser 2019), (Dunnmon et al 2019), (Perez et al 2019)). In this paper, we demonstrate the use of CNNs for predicting food security metrics in poor countries in Africa using a combination of publicly available satellite imagery data from the work of (Saikat Basu 2015) ("DeepSat" dataset) and UN data made available by the Stanford Sustainability Lab ("SustLab" dataset).…”

Section: Introductionmentioning

confidence: 99%

Predicting Food Security Outcomes Using Convolutional Neural Networks (CNNs) for Satellite Tasking

Ganguli,

Dunnmon,

Hau

2019

Preprint

Self Cite

View full text Add to dashboard Cite

Obtaining reliable data describing local Food Security Metrics (FSM) at a granularity that is informative to policy-makers requires expensive and logistically difficult surveys, particularly in the developing world. We train a CNN on publicly available satellite data describing land cover classification and use both transfer learning and direct training to build a model for FSM prediction purely from satellite imagery data. We then propose efficient tasking algorithms for high resolution satellite assets via transfer learning, Markovian search algorithms, and Bayesian networks.

show abstract

Semi-Supervised Multitask Learning on Multispectral Satellite Images Using Wasserstein Generative Adversarial Networks (GANs) for Predicting Poverty

Cited by 8 publications

References 22 publications

Learning to segment from misaligned and partial labels

Learning to segment from misaligned and partial labels

GeoGAN: A Conditional GAN with Reconstruction and Style Loss to Generate Standard Layer of Maps from Satellite Images

Predicting Food Security Outcomes Using Convolutional Neural Networks (CNNs) for Satellite Tasking

Contact Info

Product

Resources

About