Semantic photo manipulation with a generative image prior

Bau, David; Strobelt, Hendrik; Peebles, William; Wulff, J.; Zhou, Bolei; Zhu, Jun‐Yan; Torralba, Antonio

doi:10.1145/3306346.3323023

Cited by 296 publications

(232 citation statements)

References 38 publications

Supporting

Mentioning

232

Contrasting

Order By: Relevance

“…Finally, for future work, we would like to exploit current deep learning techniques to our problem. Recently developed generative adversarial networks (GAN) have been successfully applied to various user guided image synthesis tasks [26,40,41]. Thus, instead of relying on the interpolation or the transfer of the existing makeup, professional and proper makeup can be generated from rough sketches of novice users, from which they can benefit by learning how to do makeup more effectively.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

Makeup Interpolation Based on Color and Shape Parametrization

2019

View full text Add to dashboard Cite

In this paper, we address the problem of synthesizing continuous variations with the appearance of makeup by taking a linear combination of the examples. Makeup usually shows a vague boundary and does not form a clear shape, which makes this problem unique from the existing image interpolation problems. We approach this problem as an interpolation between semi-transparent image layers and tackle this by presenting new parametrization schemes for the color and for the shape separately in order to achieve an effective interpolation. For the color parametrization, our main idea is based on the observation of the symmetric relation between the color and transparency of the makeup; we provide an optimization framework for extracting a representative palette of colors associated with the transparent values, which enables us to easily set up the color correspondence among the multiple makeup samples. For the shape parametrization, we exploit a polar coordinate system, that creates the in-between shapes effectively, without ghosting artifacts.

show abstract

Section: Conclusion and Discussionmentioning

confidence: 99%

Makeup Interpolation Based on Color and Shape Parametrization

2019

View full text Add to dashboard Cite

show abstract

“…Since their introduction in 2014, Generative Adversarial Networks (GANs) [14] have been successfully applied to various image synthesis tasks, e.g. image inpainting [48,11], image manipulation [52,5,1] and texture synthesis [29,43,12]. With continuous improvements on GAN architecture [40,25,38], loss function [33,4] and regularization [16,36,34], the images synthesized by GANs are becoming more and more realistic.…”

Section: Related Workmentioning

confidence: 99%

SEAN: Image Synthesis With Semantic Region-Adaptive Normalization

Zhu

Abdal

Qin

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

422

381

View full text Add to dashboard Cite

Style Image (a) (b) (c) (d) (e) (f) Figure 1: Face image editing controlled via style images and segmentation masks. a) source images. b) reconstruction of the source image; segmentation mask shown as small inset. c-f) four separate edits; we show the image that provides new style information on top and show the part of the segmentation mask that gets edited as small inset. The results of the successive edits are shown in row two and three. The four edits change hair, mouth and eyes, skin tone, and background, respectively.

show abstract

“…Image manipulation: Modifying images through some userdefined conditions is a challenging task. Most of the previous approaches rely on conditional inpaiting [2,17,46], in which the network fills a user-selected area with pixel values coherent with the user preferences and the context. However, image editing does not necessarily require to select the exact pixels that have to be changed.…”

Section: Input Text Featuresmentioning

confidence: 99%

Describe What to Change

Liu

Nadai

Cai

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Manipulating visual attributes of images through human-written text is a very challenging task. On the one hand, models have to learn the manipulation without the ground truth of the desired output. On the other hand, models have to deal with the inherent ambiguity of natural language. Previous research usually requires either the user to describe all the characteristics of the desired image or to use richly-annotated image captioning datasets. In this work, we propose a novel unsupervised approach, based on imageto-image translation, that alters the attributes of a given image through a command-like sentence such as "change the hair color to black". Contrarily to state-of-the-art approaches, our model does not require a human-annotated dataset nor a textual description of all the attributes of the desired image, but only those that have to be modified. Our proposed model disentangles the image content from the visual attributes, and it learns to modify the latter using the textual description, before generating a new image from the content and the modified attribute representation. Because text might be inherently ambiguous (blond hair may refer to different shadows of blond, e.g. golden, icy, sandy), our method generates multiple stochastic versions of the same translation. Experiments show that the proposed model achieves promising performances on two large-scale public datasets: CelebA and CUB. We believe our approach will pave the way to new avenues of research combining textual and speech commands with visual attributes. CCS CONCEPTS • Computing methodologies → Computer vision; Machine learning.

show abstract

Semantic photo manipulation with a generative image prior

Cited by 296 publications

References 38 publications

Makeup Interpolation Based on Color and Shape Parametrization

Makeup Interpolation Based on Color and Shape Parametrization

SEAN: Image Synthesis With Semantic Region-Adaptive Normalization

Describe What to Change

Contact Info

Product

Resources

About