Style and Content Disentanglement in Generative Adversarial Networks

Abstract

Disentangling factors of variation within data has become a very challenging problem for image generation tasks. Current frameworks for training a Generative Adversarial Network (GAN), learn to disentangle the representations of the data in an unsupervised fashion and capture the most significant factors of the data variations. However, these approaches ignore the principle of content and style disentanglement in image generation, which means their learned latent code may alter the content and style of the generated images at the same time. This paper describes the Style and Content Disentangled GAN (SC-GAN), a new unsupervised algorithm for training GANs that learns disentangled style and content representations of the data. We assume that the representation of an image can be decomposed into a content code that represents the geometrical information of the data, and a style code that captures textural properties. Consequently, by fixing the style portion of the latent representation, we can generate diverse images in a particular style. Reversely, we can set the content code and generate a specific scene in a variety of styles. The proposed SC-GAN has two components: a content code which is the input to the generator, and a style code which modifies the scene style through modification of the Adaptive Instance Normalization (AdaIN) layers' parameters. We evaluate the proposed SC-GAN framework on a set of baseline datasets.

Arbitrary Style Transfer via Multi-Adaptation Network

“…Feature Disentanglement. In recent years, researches [5,7,11,13,14,19,29,31,32] used generative adversarial networks (GANs), which can be applied to style transfer task in some cases, to achieve image-to-image translation. A significant thought suitable for the style transfer task is that the style and content feature should be disentangled because of the domain deviation.…”

“…Kotovenko et al [13] proposed a disentanglement loss to separate style and content. Kazemi et al [11] described a style and content disentangled GAN (SC-GAN) to learn a semantic representation of content and textual patterns of style. Yu et al [29] disentangled the input to latent code through an encoder-decoder network.…”

“…In recent years, some researchers [5,7,11,13,14,19,26,29,31,32] have used Generative Adversarial Networks (GANs) for high-quality image-to-image translation. GAN-based methods can generate high-quality artistic works that can be considered as real.…”

Simultaneous Multi‐Attribute Image‐to‐Image Translation Using Parallel Latent Transform Networks

“…Lee et al [LTH * 18] divide the latent space to a content space and a domain-specific attribute space, which helps to keep the content unchanged when changing the image style. Kazemi et al [KIN19] also learn to disentangle the representations of style from content of the data. Since these methods only focus on a certain aspect of the representation, they generally cannot be used to transfer multiple specified attributes simultaneously.…”

Intelligent Transportation Systems, Hybrid Electric Vehicles, Powertrain Control, Cooperative Adaptive Cruise Control, Model Predictive Control

“…The main goal of artificial intelligence (AI) is to approach complicated tasks with raw sensory inputs. In recent years, deep learning techniques have quickly become the state of the art in most of the AI realted applications, specially in computer vision [107,108,109,110,111,112,113,114,115].…”