ColorMapGAN: Unsupervised Domain Adaptation for Semantic Segmentation Using Color Mapping Generative Adversarial Networks

Abstract

Due to the various reasons such as atmospheric effects and differences in acquisition, it is often the case that there exists a large difference between spectral bands of satellite images collected from different geographic locations. The large shift between spectral distributions of training and test data causes the current state of the art supervised learning approaches to output unsatisfactory maps. We present a novel semantic segmentation framework that is robust to such shift. The key component of the proposed framework is Color Mapping Generative Adversarial Networks (ColorMapGAN), which can generate fake training images that are semantically exactly the same as training images, but whose spectral distribution is similar to the distribution of the test images. We then use the fake images and the ground-truth for the training images to fine-tune the already trained classifier. Contrary to the existing Generative Adversarial Networks (GANs), the generator in ColorMapGAN does not have any convolutional or pooling layers. It learns to transform the colors of the training data to the colors of the test data by performing only one element-wise matrix multiplication and one matrix addition operations. Thanks to the architecturally simple but powerful design of ColorMapGAN, the proposed framework outperforms the existing approaches with a large margin in terms of both accuracy and computational complexity.

Cross-Sensor Adversarial Domain Adaptation of Landsat-8 and Proba-V Images for Cloud Detection

“…Table II shows the results in the target domain (Proba-V) using the PV24 dataset with the trained DA transformation G PV→LU (called full DA in the table) and without it (called no DA). We also included the results of the ablation study, where we have set some of the weights of the generator losses to zero and the results using histogram matching [36] for domain adaptation as in [47]. In addition, results are compared with the FCNN trained in original Proba-V images and ground truths (PV-trained), which serves as an upper bound reference, and with the operational Proba-V cloud detection algorithm (v101) [27].…”

Improving Land Cover Segmentation Across Satellites Using Domain Adaptation

“…However, because RS is a vast field, it is tricky to classify such studies. As described by Tuia et al [37], some methods are based on selecting invariant features on the training data [38], [39], whereas others are based on the adaptation of the data distribution [40]- [42], and lastly building the adaptation in the classifier. The first type of method may be time-consuming because of the difficulty of selecting invariant data, which might require building a new dataset for each target data point.…”

Data-Efficient Domain Adaptation for Semantic Segmentation of Aerial Imagery Using Generative Adversarial Networks

“…Then, the translated dataset is used to fine-tune the segmentation model to enhance its ability to treat images from the target domain. The second work is the work proposed by Tasar et al [37]. They adopted the same algorithm of Benjdira et al [4] but changed the GAN architecture into another architecture named ColorMapGAN.…”