Multi-Style Unsupervised Image Synthesis Using Generative Adversarial Nets

Abstract

Unsupervised cross-domain image-to-image translation is a very active topic in computer vision and graphics. This task has two challenges: 1) lack of paired training data and 2) numerous possible outputs from a single image. The existing methods rely on either paired data or perform one-to-one translation. A novel Multi-Style Unsupervised image synthesis model using Generative Adversarial Nets (MSU-GAN) is proposed in this paper to overcome these disadvantages. Firstly, the encoder-decoder structure is used to map the image to domain-shared content features space and domain-specific style features space. Secondly, to translate an image into another domain, the content code and the style code are combined to synthesize the resulting image. Finally, the bidirectional cycle-consistency loss is used for the unpaired training data; the inter-domain adversarial loss and the reconstruction loss are used to ensure the output image's realism. Simultaneously, MSU-GAN is able to synthesize multi-style images due to disentangled representation. A Multi-Style Unsupervised Feature-Wise image synthesis model using Generative Adversarial Nets (MSU-FW-GAN) based on the MSU-GAN is proposed for the shape variation tasks. There are two different testing strategies, which include random style transfer and style guide transfer. For objective comparison, the proposed model performs well on all evaluation metrics. The random style transfer experiment results show that compared with CycleGAN on the photo2portraits dataset, MSU-FW-GAN FID, IS scores dropped by 12.77% and 8.06%. For the summer2winter dataset, MSU-GAN FID and IS scores increased by 24.51% and 3.64%. Qualitative results show that without paired training data, MSU-GAN and MSU-FW-GAN can synthesize multi-style and better realistic images on various tasks.

Generation and Evaluation of Tabular Data in Different Domains Using Gans

“…And then plotted the autocorrelation plot for visualization. We then used the Xgb model to check if there is overfitting using the validation and training errors [19][20][21][22].…”

Asymmetric Two-Stage CycleGAN for Generation of Faces With Shadow Puppet Style

“…For example, Gao et al [15] decomposed image stylization into four steps: feature encoding, feature destylization, feature restylization, and feature decoding, and the model's capabilities of retaining contents and capturing style were improved. Lv et al [16] and Lee et al [17] resolved image transformation into two aspects, style and contents, and encoded them separately, subsequently, the generation process takes both aspects as input. Liu et al [18] proposed UNIT, which can transform different scenes, species, and faces.…”

A Novel Method for Road Anomaly Objects Detection in the Traffic Environment With Multi-Mechanism Fusion

“…However, this method still lacks the ability to accurately distinguish between known classes and anomalies. The second method involves resynthesizing a new input image from the semantic map predicted by the segmentation network, and detecting anomalies by analyzing the feature differences among the original input image, the predicted semantic map, and the synthesized image [11]. This method exhibits a high extraction effect when dealing with objects outside the network's training classification but introduces a problem that cannot be ignored.…”