Gated-GAN: Adversarial Gated Networks for Multi-Collection Style Transfer

Abstract

Style transfer describes the rendering of an image's semantic content as different artistic styles. Recently, generative adversarial networks (GANs) have emerged as an effective approach in style transfer by adversarially training the generator to synthesize convincing counterfeits. However, traditional GAN suffers from the mode collapse issue, resulting in unstable training and making style transfer quality difficult to guarantee. In addition, the GAN generator is only compatible with one style, so a series of GANs must be trained to provide users with choices to transfer more than one kind of style. In this paper, we focus on tackling these challenges and limitations to improve style transfer. We propose adversarial gated networks (Gated-GAN) to transfer multiple styles in a single model. The generative networks have three modules: an encoder, a gated transformer, and a decoder. Different styles can be achieved by passing input images through different branches of the gated transformer. To stabilize training, the encoder and decoder are combined as an auto-encoder to reconstruct the input images. The discriminative networks are used to distinguish whether the input image is a stylized or genuine image. An auxiliary classifier is used to recognize the style categories of transferred images, thereby helping the generative networks generate images in multiple styles. In addition, Gated-GAN makes it possible to explore a new style by investigating styles learned from artists or genres. Our extensive experiments demonstrate the stability and effectiveness of the proposed model for multi-style transfer.

Robust Nonparametric Distribution Transfer with Exposure Correction for Image Neural Style Transfer

“…As a form of preprocessing, the proposed model can improve the performances of the corresponding style transfer method without depending on any specific neural style transfer model. To evaluate this, two quantitative metrics were adopted, which are deception rate [ 52 ] and FID core [ 53 ]. As defined in [ 52 ], deception rate is proposed for an automatic evaluation of style transfer results.…”

“…It is calculated as the fraction of generated images which were classified by the network as the artworks of an artist for which the stylization was produced. FID score evaluates the style transfer results by measuring the distance between the generated distribution and the real distribution [ 53 ]. The higher deception rate means the better performance, while the lower FID score represents the better performance.…”

Semi-Supervised FaceGAN for Face-Age Progression and Regression with Synthesized Paired Images

“…Instead of generating images from random noise, conditional GAN generates an image from a given label [22] or image [23,24]. The conditional GAN is good at image-to-image translation tasks such as style transfer [25], super-resolution [26], and colorization [27]. Therefore, many studies exploited conditional GAN to synthesize aged faces.…”

Smooth Deep Image Generator from Noises

“…Convincing images generated from noisy vectors through GANs could be employed to augment image datasets, which would alleviate the shortage of training data in some tasks. Moreover, image-to-image translation (Chen et al 2018;2019) based on GANs also gets its popularity.…”