Image Processing Using Multi-Code GAN Prior

Gu, Jinjin; Shen, Yujun; Zhou, Bolei

doi:10.1109/cvpr42600.2020.00308

Cited by 303 publications

(235 citation statements)

References 25 publications

Supporting

Mentioning

232

Contrasting

Order By: Relevance

“…Pix2PixHD [28] extends it to synthesize high-resolution results. J. Gu et al [29] incorporate well-trained GANs to a variety of image processing tasks. However, it isn't easy to obtain a pair of images in an actual situation.…”

Section: A Generator Adversarial Networkmentioning

confidence: 99%

Multi-Style Unsupervised Image Synthesis Using Generative Adversarial Nets

Israr

2021

IEEE Access

View full text Add to dashboard Cite

Unsupervised cross-domain image-to-image translation is a very active topic in computer vision and graphics. This task has two challenges: 1) lack of paired training data and 2) numerous possible outputs from a single image. The existing methods rely on either paired data or perform one-to-one translation. A novel Multi-Style Unsupervised image synthesis model using Generative Adversarial Nets (MSU-GAN) is proposed in this paper to overcome these disadvantages. Firstly, the encoder-decoder structure is used to map the image to domain-shared content features space and domain-specific style features space. Secondly, to translate an image into another domain, the content code and the style code are combined to synthesize the resulting image. Finally, the bidirectional cycle-consistency loss is used for the unpaired training data; the inter-domain adversarial loss and the reconstruction loss are used to ensure the output image's realism. Simultaneously, MSU-GAN is able to synthesize multi-style images due to disentangled representation. A Multi-Style Unsupervised Feature-Wise image synthesis model using Generative Adversarial Nets (MSU-FW-GAN) based on the MSU-GAN is proposed for the shape variation tasks. There are two different testing strategies, which include random style transfer and style guide transfer. For objective comparison, the proposed model performs well on all evaluation metrics. The random style transfer experiment results show that compared with CycleGAN on the photo2portraits dataset, MSU-FW-GAN FID, IS scores dropped by 12.77% and 8.06%. For the summer2winter dataset, MSU-GAN FID and IS scores increased by 24.51% and 3.64%. Qualitative results show that without paired training data, MSU-GAN and MSU-FW-GAN can synthesize multi-style and better realistic images on various tasks.

show abstract

Section: A Generator Adversarial Networkmentioning

confidence: 99%

Multi-Style Unsupervised Image Synthesis Using Generative Adversarial Nets

Israr

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…On the CK+ dataset, we do the image quality evaluation, two indicators are used to quantitatively evaluate the image quality, they are peak signal-to-noise ratio (PSNR) and structure similarity (SSIM). And our method is based on GAN, several evaluations on perceptual metrics like LPIPS [36], FID [37] are also widely used in image synthesis tasks [38], [39]. Here we chooses one more indicator FID which will be helpful to give a more objective evaluation.…”

Section: ) Ck+ Datasetmentioning

confidence: 99%

Facial Landmarks and Expression Label Guided Photorealistic Facial Expression Synthesis

Sun

2021

IEEE Access

View full text Add to dashboard Cite

Facial expression manipulation plays an increasingly important role in the field of computer graphics and has been widely used in generating facial animations. However, it is still a very challenging task as it needs full understanding of the input face and very depending on the facial appearance. In this paper, we present an end-to-end generative adversarial network for facial expression synthesis. Given the facial landmarks and the expression label of a target image, our method automatically generates a corresponding expression facial image with the identity information and facial details well preserved. Both qualitative and quantitative experiments are conducted on the CK+ and Oulu-CASIA datasets. Experimental results show that our method has the compelling perceptual results even there exist large differences in facial shapes for unseen subjects.INDEX TERMS Facial expression synthesis, generative adversarial networks.

show abstract

“…Gu et al . [GSZ20] proposed a new inversion method that operate on multiple latent vectors instead of one. Their novel inversion method enable multiple image processing applications such as image colourization, inpainting and so on.…”

Section: Related Workmentioning

confidence: 99%

“…Although the recent proposed iterative inversion methods obtain better reconstruction results compared to iGAN, we argue that compares to iGAN is more suitable since iGAN combines iterative optimization inversion with user controls. The other methods requires additional training for each input image [BSP*19], decision of the number of latent codes [GSZ20] or tailored latent space for exploration [AQW20]. Nevertheless, these advances on using different loss functions or optimizers can benefit the interactive methods including our method and iGAN.…”

Section: Related Workmentioning

confidence: 99%

Interactive Optimization of Generative Image Modelling using Sequential Subspace Search and Content‐based Guidance

Chong

Shen

Sato

et al. 2020

Computer Graphics Forum

View full text Add to dashboard Cite

Generative image modeling techniques such as GAN demonstrate highly convincing image generation result. However, user interaction is often necessary to obtain desired results. Existing attempts add interactivity but require either tailored architectures or extra data. We present a human‐in‐the‐optimization method that allows users to directly explore and search the latent vector space of generative image modelling. Our system provides multiple candidates by sampling the latent vector space, and the user selects the best blending weights within the subspace using multiple sliders. In addition, the user can express their intention through image editing tools. The system samples latent vectors based on inputs and presents new candidates to the user iteratively. An advantage of our formulation is that one can apply our method to arbitrary pre‐trained model without developing specialized architecture or data. We demonstrate our method with various generative image modelling applications, and show superior performance in a comparative user study with prior art iGAN [ZKSE16].

show abstract

Image Processing Using Multi-Code GAN Prior

Cited by 303 publications

References 25 publications

Multi-Style Unsupervised Image Synthesis Using Generative Adversarial Nets

Multi-Style Unsupervised Image Synthesis Using Generative Adversarial Nets

Facial Landmarks and Expression Label Guided Photorealistic Facial Expression Synthesis

Interactive Optimization of Generative Image Modelling using Sequential Subspace Search and Content‐based Guidance

Contact Info

Product

Resources

About