Aerial Imagery Feature Engineering Using Bidirectional Generative Adversarial Networks: A Case Study of the Pilica River Region, Poland

Abstract

Generative adversarial networks (GANs) are a type of neural network that are characterized by their unique construction and training process. Utilizing the concept of the latent space and exploiting the results of a duel between different GAN components opens up interesting opportunities for computer vision (CV) activities, such as image inpainting, style transfer, or even generative art. GANs have great potential to support aerial and satellite image interpretation activities. Carefully crafting a GAN and applying it to a high-quality dataset can result in nontrivial feature enrichment. In this study, we have designed and tested an unsupervised procedure capable of engineering new features by shifting real orthophotos into the GAN’s underlying latent space. Latent vectors are a low-dimensional representation of the orthophoto patches that hold information about the strength, occurrence, and interaction between spatial features discovered during the network training. Latent vectors were combined with geographical coordinates to bind them to their original location in the orthophoto. In consequence, it was possible to describe the whole research area as a set of latent vectors and perform further spatial analysis not on RGB images but on their lower-dimensional representation. To accomplish this goal, a modified version of the big bidirectional generative adversarial network (BigBiGAN) has been trained on a fine-tailored orthophoto imagery dataset covering the area of the Pilica River region in Poland. Trained models, precisely the generator and encoder, have been utilized during the processes of model quality assurance and feature engineering, respectively. Quality assurance was performed by measuring model reconstruction capabilities and by manually verifying artificial images produced by the generator. The feature engineering use case, on the other hand, has been presented in a real research scenario that involved splitting the orthophoto into a set of patches, encoding the patch set into the GAN latent space, grouping similar patches latent codes by utilizing hierarchical clustering, and producing a segmentation map of the orthophoto.

Generative adversarial approach to urban areas NDVI estimation: A case study of Łódź, Poland

“…It is a common practice to extract a trained and finetuned GANs subnet to apply it to solve a certain research problem. frequently, the generator is the target of such a procedure but there are also cases of using only the encoder (Adamiak et al 2021). The model architecture complexity is also a reason for multiple problems that one must face when training a GAN: vanishing gradient, mode collapse and convergence failure are the most common.…”

The use of deep learning technology in dance movement generation

“…The other is the generative network. The goal is to generate real images as much as possible to deceive the discriminative network and make it unable to distinguish the samples from the source (Adamiak et al, 2021;Jeong et al, 2021). The final ideal result is that the model converges, and the discriminative network cannot judge the authenticity of the input samples, that is, the generated network can generate samples in line with the real data distribution.…”

Generative Adversarial Approach to Urban Areas NDVI Estimation Using Information Exclusively from Structural and Textural Analysis of Panchromatic Orthoimagery: A Case Study of Łódź, Poland

“…It is a common practice to extract a trained and fine-tuned GANs subnet to apply it to solve a certain research problem. Frequently, the generator is the target of such a procedure but there are also cases of using only the encoder [57]. The model architecture complexity is also a reason for multiple problems that one must face when training a GAN: vanishing gradient, mode collapse and convergence failure are the most common.…”