RetrieveGAN: Image Synthesis via Differentiable Patch Retrieval

Tseng, Hung Yu; Lee, Hsin Ying; Jiang, Lu; Yang, Ming–Hsuan; Yang, Wei

doi:10.1007/978-3-030-58598-3_15

Cited by 37 publications

(29 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The core idea of GANs is a two-player game between a generator aiming to map noise vectors to realistic images and a discriminator attempting to discriminate the generated images from the real ones. GANs facilitate a variety of creation tasks such as imageto-image translation [48], text-to-image generation [33,43], semantic image synthesis [7,27], video generation [20,35], etc. However, most of the models generate new images from scratch given various conditional contexts, and generally lack the ability to perform editing and interactive manipulation on existing images.…”

Section: Related Workmentioning

confidence: 99%

In&Out : Diverse Image Outpainting via GAN Inversion

Cheng¹,

Lin²,

Lee³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Section: Related Workmentioning

confidence: 99%

In&Out : Diverse Image Outpainting via GAN Inversion

Cheng¹,

Lin²,

Lee³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

“…The VAE [26] and GAN [5] nowadays provide the backbone for image generation tasks such as image reconstruction [27][28][29][30], image synthesis [8,31,32], and image translation [33][34][35]. In a VAE, the encoder maps images into a latent feature space which is then mapped back to the image domain through a decoder.…”

Section: Image Generation Algorithmmentioning

confidence: 99%

Mask-aware photorealistic facial attribute manipulation

et al. 2021

View full text Add to dashboard Cite

The technique of facial attribute manipulation has found increasing application, but it remains challenging to restrict editing of attributes so that a face’s unique details are preserved. In this paper, we introduce our method, which we call a mask-adversarial autoencoder (M-AAE). It combines a variational autoencoder (VAE) and a generative adversarial network (GAN) for photorealistic image generation. We use partial dilated layers to modify a few pixels in the feature maps of an encoder, changing the attribute strength continuously without hindering global information. Our training objectives for the VAE and GAN are reinforced by supervision of face recognition loss and cycle consistency loss, to faithfully preserve facial details. Moreover, we generate facial masks to enforce background consistency, which allows our training to focus on the foreground face rather than the background. Experimental results demonstrate that our method can generate high-quality images with varying attributes, and outperforms existing methods in detail preservation.

show abstract

“…The authors of [37] apply attention on edges with the specific goal of graph clustering. Applications range from (but are not limited to) recommender systems [38], pose estimation [39], video classification [40], event detection [41], and image synthesis [42]. The authors of [43] apply attention on features of discrete input samples and an attacking vocabulary, in order to acquire a scalable method for real-time generation of adversarial examples on discrete input data.…”

Section: Attentionmentioning

confidence: 99%

A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning

Huijben¹,

Kool²,

Paulus³

et al. 2021

Preprint

View full text Add to dashboard Cite

The Gumbel-max trick is a method to draw a sample from a categorical distribution, given by its unnormalized (log-)probabilities. Over the past years, the machine learning community has proposed several extensions of this trick to facilitate, e.g., drawing multiple samples, sampling from structured domains, or gradient estimation for error backpropagation in neural network optimization. The goal of this survey article is to present background about the Gumbel-max trick, and to provide a structured overview of its extensions to ease algorithm selection. Moreover, it presents a comprehensive outline of (machine learning) literature in which Gumbel-based algorithms have been leveraged, reviews commonly-made design choices, and sketches a future perspective.

show abstract

RetrieveGAN: Image Synthesis via Differentiable Patch Retrieval

Cited by 37 publications

References 38 publications

In&Out : Diverse Image Outpainting via GAN Inversion

In&Out : Diverse Image Outpainting via GAN Inversion

Mask-aware photorealistic facial attribute manipulation

A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning

Contact Info

Product

Resources

About