Constructing Self-Motivated Pyramid Curriculums for Cross-Domain Semantic Segmentation: A Non-Adversarial Approach

Abstract

We propose a new approach, called self-motivated pyramid curriculum domain adaptation (PyCDA), to facilitate the adaptation of semantic segmentation neural networks from synthetic source domains to real target domains. Our approach draws on an insight connecting two existing works: curriculum domain adaptation and self-training. Inspired by the former, PyCDA constructs a pyramid curriculum which contains various properties about the target domain. Those properties are mainly about the desired label distributions over the target domain images, image regions, and pixels. By enforcing the segmentation neural network to observe those properties, we can improve the network's generalization capability to the target domain. Motivated by the self-training, we infer this pyramid of properties by resorting to the semantic segmentation network itself. Unlike prior work, we do not need to maintain any additional models (e.g., logistic regression or discriminator networks) or to solve min max problems which are often difficult to optimize. We report state-of-the-art results for the adaptation from both GTAV and SYNTHIA to Cityscapes, two popular settings in unsupervised domain adaptation for semantic segmentation.

Content-Consistent Matching for Domain Adaptive Semantic Segmentation

“…Compared with methods established on self-training, CCM achieves comparable or even better results. For example, our method is on par with pyCDA [28], i.e. 49.9% (ours) vs. 47.4% (pyCDA) on GTA5 → Cityscapes and 45.2%/52.9% (mIoU and mIoU* of ours) vs. 46.7%/53.3% (mIoU and mIoU* of pyCDA) on SYNTHIA → Cityscapes.…”

“…Some recent approaches adopted self-training to perform adaptation. [58] proposed to assign pseudo labels in a curriculum way and [50,59,28,55] combined self-training with other constraints to improve the quality of pseudo labels.…”

“…In both tables, we use "AT" and "ST" to denote approaches established on adversarial training and self-training respectively, while "AS" indicates methods utilizing both. All the models are based on DeepLabV2-Res101 backbone, except that pyCDA [28] is based on PSPNet [54] and CAG [50] is based on DeepLabV3+ [6] 4 . It can be seen that our method outperforms source only baseline with a large margin, which verifies the effectiveness of our approach.…”

“…And the training tries to utilize the knowledge learned from the source and mitigate the domain shift. Previous methods usually achieved the adapted model through the adversarial training [41,44,34,24,13,18,2,27] or by self-training [58,59,28,50]. All those methods employed the entire source domain data throughout the training process, which neglects the fact that not all the source images could contribute to the improvement of adaptation performance, especially at certain training stages.…”

InjectionGAN: Unified Generative Adversarial Networks for Arbitrary Image Attribute Editing

“…There are many improvements based on the GAN model. DCGAN [28] refined the design of the network of generators and discriminators, which facilitated the application of GANs in many image generation tasks. cGAN [29] modifies GAN from unsupervised learning to semi-supervised learning by taking conditional variable as input to the generator and discriminator to generate image with desired properties.…”