Conditional generative adversarial networks for data augmentation and adaptation in remotely sensed imagery

Howe, Jonathan; Pula, Kyle

doi:10.1117/12.2529586

Cited by 15 publications

(15 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ma et al [53] proposed a generative adversarial network (GAN) to synthesize images for scene labeling. Howe et al [54] introduced another GAN-based approach for Earth's surface object detection in airborne images. Zheng et al [55] proposed a method for generating synthetic vehicles in aerial images.…”

Section: Related Workmentioning

confidence: 99%

Cloud and Cloud Shadow Segmentation for Remote Sensing Imagery Via Filtered Jaccard Loss Function and Parametric Augmentation

Mohajerani

Saeedi

2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Cloud and cloud shadow segmentation are fundamental processes in optical remote sensing image analysis. Current methods for cloud/shadow identification in geospatial imagery are not as accurate as they should, especially in the presence of snow and haze. This paper presents a deep learningbased framework for the detection of cloud/shadow in Landsat 8 images. Our method benefits from a convolutional neural network, Cloud-Net+ (a modification of our previously proposed Cloud-Net [1]) that is trained with a novel loss function (Filtered Jaccard Loss). The proposed loss function is more sensitive to the absence of foreground objects in an image and penalizes/rewards the predicted mask more accurately than other common loss functions. In addition, a sunlight direction-aware data augmentation technique is developed for the task of cloud shadow detection to extend the generalization ability of the proposed model by expanding existing training sets. The combination of Cloud-Net+, Filtered Jaccard Loss function, and the proposed augmentation algorithm delivers superior results on four public cloud/shadow detection datasets. Our experiments on Pascal VOC dataset exemplifies the applicability and quality of our proposed network and loss function in other computer vision applications.

show abstract

Section: Related Workmentioning

confidence: 99%

Cloud and Cloud Shadow Segmentation for Remote Sensing Imagery Via Filtered Jaccard Loss Function and Parametric Augmentation

Mohajerani

Saeedi

2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Scene layout generation. SB-GAN [2] and PGAN-CGAN [13] were the first GAN-based approaches proposed for this task. The SB-GAN pipeline combines an uncon-ditional model based on ProGAN [15] for generating the semantic masks and GauGAN [26] for transforming these masks into photo-realistic images.…”

Section: Related Workmentioning

confidence: 99%

“…Hence, generating this type of data with a GAN amounts to producing matching image-layout pairs. To this end, recent works advocate decoupling the synthesis process into two consecutive phases: first generating semantic layouts with plausible object arrangements [2,13], then translating these layouts into realistic images [26,35].…”

Section: Introductionmentioning

confidence: 99%

“…To improve the usability of such a pipeline, we mostly focus here on the first layout generation step. Existing works [2,13] cast it as a standard generative process that turns a random input code into a semantic map. While simple to use, this approach offers no real control on the modes of the output distribution [24], which is a limitation for complex scenes.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Semantic Palette: Guiding Scene Generation with Class Proportions

Moing¹,

Vu²,

Jain³

et al. 2021

Preprint

View full text Add to dashboard Cite

Despite the recent progress of generative adversarial networks (GANs) at synthesizing photo-realistic images, producing complex urban scenes remains a challenging problem. Previous works break down scene generation into two consecutive phases: unconditional semantic layout synthesis and image synthesis conditioned on layouts. In this work, we propose to condition layout generation as well for higher semantic control: given a vector of class proportions, we generate layouts with matching composition. To this end, we introduce a conditional framework with novel architecture designs and learning objectives, which effectively accommodates class proportions to guide the scene generation process. The proposed architecture also allows partial layout editing with interesting applications. Thanks to the semantic control, we can produce layouts close to the real distribution, helping enhance the whole scene generation process. On different metrics and urban scene benchmarks, our models outperform existing baselines. Moreover, we demonstrate the merit of our approach for data augmentation: semantic segmenters trained on real layoutimage pairs along with additional ones generated by our approach outperform models only trained on real pairs.

show abstract

“…These methods train the neural network directly with the simulation dataset without thinking that the trained networks may not suitable for the actual dataset and may not achieve a promising result. Howe et al [5] proposed a novel data augmentation strategy based on simulated samples object detection in remote sensing images. These methods consider the problem of how to match the target object with the background image in terms of size, tilt angle, and image resolution.…”

Section: Introductionmentioning

confidence: 99%

Target Detection Based on Simulated Image Domain Migration

Wang

Zhao

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Annotating a large amount of data manually for supervised learning is an indispensable and expensive part. A novel system using the simulation dataset is proposed in this paper. This framework can train the neural networks for remote sensing object detection without any manually labeled dataset. The whole system can be divided into three parts. The first part is the dataset simulator. The simulator synthesizes remote sensing images with the aircraft targets based on real remote sensing images (without any aircraft targets). In the process of data generation, the simulator automatically marks the position information of the aircraft. The second part is the image dataset domain adaptation work. We introduce the work of Cycle-GAN into this part to bridge the perceptual gap between the simulation dataset and reality dataset. Specially, we propose a multi-scale generator into the original Cycle-GAN model to achieve better domain adaptation performance. The final part is the object detection neural network. The domain adaptation quality of the remote sensing images reconstructed by our novel cycle-gan network achieves better performance both in the structural similarity measure and visual appearance. The object detection model trained with the dataset processed by our novel system can get better detection precision. The analytic experiments on the test dataset demonstrate that the object detection model trained with the dataset processed by our novel system can get better detection precision. INDEX TERMS Remote sensing image, generative and adversarial network, object detection, data simulation.

show abstract

Conditional generative adversarial networks for data augmentation and adaptation in remotely sensed imagery

Cited by 15 publications

References 27 publications

Cloud and Cloud Shadow Segmentation for Remote Sensing Imagery Via Filtered Jaccard Loss Function and Parametric Augmentation

Cloud and Cloud Shadow Segmentation for Remote Sensing Imagery Via Filtered Jaccard Loss Function and Parametric Augmentation

Semantic Palette: Guiding Scene Generation with Class Proportions

Target Detection Based on Simulated Image Domain Migration

Contact Info

Product

Resources

About