Toward learning a unified many-to-many mapping for diverse image translation

Abstract

Image-to-image translation, which translates input images to a different domain with a learned one-to-one mapping, has achieved impressive success in recent years. The success of translation mainly relies on the network architecture to reserve the structural information while modify the appearance slightly at the pixel level through adversarial training. Although these networks are able to learn the mapping, the translated images are predictable without exclusion. It is more desirable to diversify them using image-to-image translation by introducing uncertainties, i.e., the generated images hold potential for variations in colors and textures in addition to the general similarity to the input images, and this happens in both the target and source domains. To this end, we propose a novel generative adversarial network (GAN) based model, InjectionGAN, to learn a many-to-many mapping. In this model, the input image is combined with latent variables, which comprise of domainspecific attribute and unspecific random variations. The domain-specific attribute indicates the target domain of the translation, while the unspecific random variations introduce uncertainty into the model. A unified framework is proposed to regroup these two parts and obtain diverse generations in each domain. Extensive experiments demonstrate that the diverse generations have high quality for the challenging image-to-image translation tasks where no pairing information of the training dataset exits. Both quantitative and qualitative results prove the superior performance of InjectionGAN over the state-of-the-art approaches.

Innovating with Artificial Intelligence: Capturing the Constructive Functional Capabilities of Deep Generative Learning

“…VAE and GAN models can manipulate human poses and their outfits [4,12], facial expressions [21,43], their gaze [59], or their face age [75]. Further, Xu et al implemented a GAN-based model for domaintransfer that can translate face avatars into images of human faces or even simple line drawings into colored pictures [71]. An encoder-decoder network can edit the color of voxel-based 3D objects like cars [73] or a GAN can modify 3D models of buildings such that these models satisfy multiple physical constraints [62].…”

A comparative study on polyp classification using convolutional neural networks

“…Deep learning models: Inspired by the success of AlexNet [16] in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2012, convolutional neural networks (CNN) have attracted a lot of attention and been successfully applied to image classification [20][21][22], object detection [4,23,24], depth estimation [25,26], image transformation [27,28], and crowd counting [29]citesajid2020plug. VGGNets [14], and GoogleNet [17], the ILSVRC winners of 2014 and 2015, proved that deeper models could significantly increase the ability of representations.…”

On Open-Set, High-Fidelity and Identity-Specific Face Transformation

“…Identity Transformation In recent years, with the emergence of deep generative models, such as the Generative Adversarial Networks (GAN) [1] and the Variational Auto-encoder (VAE) [2,3], researchers have made tremendous progress in building deep networks for image generation [38]. Among them, face transformation is a critical task, owing to its wide realworld applications.…”