GeoGAN: A Conditional GAN with Reconstruction and Style Loss to Generate Standard Layer of Maps from Satellite Images

Abstract

Automatically generating maps from satellite images is an important task. There is a body of literature which tries to address this challenge. We created a more expansive survey of the task by experimenting with different models and adding new loss functions to improve results. We created a database of pairs of satellite images and the corresponding map of the area. Our model translates the satellite image to the corresponding standard layer map image using three main model architectures: (i) a conditional Generative Adversarial Network (GAN) which compresses the images down to a learned embedding, (ii) a generator which is trained as a normalizing flow (RealNVP) model, and (iii) a conditional GAN where the generator translates via a series of convolutions to the standard layer of a map and the discriminator input is the concatenation of the real/generated map and the satellite image. Model (iii) was by far the most promising of three models. To improve the results we also added a reconstruction loss and style transfer loss in addition to the GAN losses. The third model architecture produced the best quality of sampled images. In contrast to the other generative model where evaluation of the model is a challenging problem. since we have access to the real map for a given satellite image, we are able to assign a quantitative metric to the quality of the generated images in addition to inspecting them visually. While we are continuing to work on increasing the accuracy of the model, one challenge has been the coarse resolution of the data which upper-bounds the quality of the results of our model. Nevertheless, as will be seen in the results, the generated map is more accurate in the features it produces since the generator architecture demands a pixel-wise image translation/pixel-wise coloring. A video presentation summarizing this paper is available at: https://youtu.be/Ur0flOX-Ji0

Representing Vector Geographic Information As a Tensor for Deep Learning Based Map Generalisation

“…In the literature, most experiments of map generation with GAN are about generating an image of a map in the style of GoogleMap from the corresponding aerial photograph, and vice versa. Isola et al (2018) proposed a generic image-to-image translation model called Pix2Pix, which was later improved by many researchers to better deal with the generation of maps (Ganguli et al, 2019;Chen et al, 2020;Zhang et al, 2020;Li et al, 2020). In their work, the scale of the styletransferred map is similar to that of the aerial photograph.…”

VAE-Info-cGAN

“…autoregressive models, Variational Autoencoders (VAEs), normalizing flow models, Generative Adversarial Networks (GANs)) used in computer vision, VAEs have an accessible learned latent space which is interpretable [11] while GANs are able to generate samples of higher visual quality [2,10]. Synthetic data has been successfully generated using VAEs and GANs for a wide variety of data modalities including images [1,3,8,12], music [6], text [28], etc. Methods for disentangling attributes in the latent space of VAEs and GANs have been proposed [3,5] while modulating the latent space has been shown to help in controlling attributes in the generated data [9,10,12].…”

Semi-Supervised Multitask Learning on Multispectral Satellite Images Using Wasserstein Generative Adversarial Networks (GANs) for Predicting Poverty

“…In addition, using convolutional neural networks and GANs on geospatial data in unsupervised or semi-supervised settings has also been of interest recently; especially in domains such as food security, cybersecurity, satellite tasking, etc. ( [13,15,16]). [12] and [40] use satellite images to study local economic activities and their correlation to global economic indicators like Gross Domestic Product (GDP), Ecosystem Services Product (ESP), etc.…”