Conditional Image-to-Image Translation

Abstract

Image-to-image translation tasks have been widely investigated with Generative Adversarial Networks (GANs) and dual learning. However, existing models lack the ability to control the translated results in the target domain and their results usually lack of diversity in the sense that a fixed image usually leads to (almost) deterministic translation result. In this paper, we study a new problem, conditional image-to-image translation, which is to translate an image from the source domain to the target domain conditioned on a given image in the target domain. It requires that the generated image should inherit some domain-specific features of the conditional image from the target domain. Therefore, changing the conditional image in the target domain will lead to diverse translation results for a fixed input image from the source domain, and therefore the conditional input image helps to control the translation results. We tackle this problem with unpaired data based on GANs and dual learning. We twist two conditional translation models (one translation from A domain to B domain, and the other one from B domain to A domain) together for inputs combination and reconstruction while preserving domain independent features. We carry out experiments on men's faces from-to women's faces translation and edges to shoes&bags translations. The results demonstrate the effectiveness of our proposed method.

Exploring Explicit Domain Supervision for Latent Space Disentanglement in Unpaired Image-to-Image Translation

“…To generate controllable translation result, Lin et al [7] decompose the image latent space into domain-independent and domain-specific feature spaces, and raise a new problem named as conditional cross-domain translation which can assign domain-specific feature for generated result by feeding a conditional image in the target domain. Similar to [7], other two works [8], [9] proposed to disentangle latent space and generate diverse translation results. Choi et al [10] further proposed a StarGAN that can perform image-to-image translations for multiple domains using only a single model.…”

“…Therefore, we can leverage such a pre-trained CNN to extract the domain-specific features of an image. Compared with works [7], [8], [9] that use two separated domain-specific feature extractors for two domain translation, we utilize the domain classifier as a general domainspecific feature extractor which can be easily generalized to multi-domain translation. With the well-defined domainspecific features, the domain-independent features can be easily obtained by feature disentanglement.…”

“…Existing multi-domain translation models, such as Star-GAN, also lack the ability to control the translated results in the target domain and their results usually lack of diversity in the sense that a fixed image usually leads to (almost) deterministic translation result. With two kinds of latent feature (i.e., domain-specific and domain-independent features) disentangled, we can devise a conditional DosGAN (briefly, DosGAN-c) for conditional image-to-image translation [7], which is to translate an image from the source domain to the target domain conditioned on a given image in the target domain and requires that the generated image should inherit some domain-specific features of the conditional image from the target domain. The DosGAN-c shares the same network architecture with DosGAN, and the difference between two frameworks only lies in the inputs and losses.…”

“…Compared with the previous work cd-GAN [7], there are three main differences: (1) Unlike cd-GAN or CycleGAN that simply treats multiple domains as different sources of images, in this work, we regard domain information as explicit supervision, where we train a classifier to classify the domain of the input image, and convert it to a domainspecific feature extractor. (2) The domain-specific features in our model are explicitly modeled, which are extracted by a well-defined and fixed domain classification network.…”

Mix and Match Networks: Encoder-Decoder Alignment for Zero-Pair Image Translation

“…Unpaired translations Various works extended this idea to the case where no explicit input-output image pairs are available (unpaired image translation), using the idea of cyclic consistency [31,72,79,41] or consistency between certain extracted features [63]. To avoid accidental artifacts and improve learning, Mejjati et al [48] integrate an attention mechanism to help translations focus on semantically meaningful regions.…”