Disentangled Person Image Generation

Ma, Liqian; Sun, Quansen; Georgoulis, Stamatios; Gool, Luc Van; Schiele, Bernt; Fritz, Mario

doi:10.1109/cvpr.2018.00018

Cited by 710 publications

(1,167 citation statements)

References 30 publications

(46 reference statements)

Supporting

Mentioning

1,150

Contrasting

Unclassified

Order By: Relevance

“…[25] to fine-grained in-painting technologies [26]. Recent approaches have been used to generate images conditioned on specific visual attributes [27], and these images range from faces [28] to people [29]. In a similar vein, Nguyen et al [30] used generative networks to create a natural-looking image that maximizes a specific neuron (the beauty neuron).…”

Section: Related Workmentioning

confidence: 99%

FaceLift: a transparent deep learning framework to beautify urban scenes

et al. 2020

View full text Add to dashboard Cite

In the area of computer vision, deep learning techniques have recently been used to predict whether urban scenes are likely to be considered beautiful: it turns out that these techniques are able to make accurate predictions. Yet they fall short when it comes to generating actionable insights for urban design. To support urban interventions, one needs to go beyond predicting beauty, and tackle the challenge of recreating beauty. Unfortunately, deep learning techniques have not been designed with that challenge in mind. Given their "black-box nature", these models cannot be directly used to explain why a particular urban scene is deemed to be beautiful. To partly fix that, we propose a deep learning framework (which we name FaceLift 1 ) that is able to both beautify existing urban scenes (Google Street views) and explain which urban elements make those transformed scenes beautiful. To quantitatively evaluate our framework, we cannot resort to any existing metric (as the research problem at hand has never been tackled before) and need to formulate new ones. These new metrics should ideally capture the presence (or absence) of elements that make urban spaces great. Upon a review of the urban planning literature, we identify five main metrics: walkability, green spaces, openness, landmarks and visual complexity. We find that, across all the five metrics, the beautified scenes meet the expectations set by the literature on what great spaces tend to be made of. This result is further confirmed by a 20-participant expert survey in which FaceLift have been found to be effective in promoting citizen participation. All this suggests that, in the future, as our framework's components are further researched and become better and more sophisticated, it is not hard to imagine technologies that will be able to accurately and efficiently support architects and planners in the design of the spaces we intuitively love.

show abstract

Section: Related Workmentioning

confidence: 99%

FaceLift: a transparent deep learning framework to beautify urban scenes

et al. 2020

View full text Add to dashboard Cite

show abstract

“…We can see that the proposed AsymmetricGAN produces much more photo-realistic results with convincing details compared with other approaches, i.e., GestureGAN [10], PG 2 [8], DPIG [26], PoseGAN [9] and SAMG [50]. Moreover, we provide quantitative comparison with those methods.…”

Section: B Hand Gesture-to-gesture Translation Taskmentioning

confidence: 79%

“…Similar ideas have also been applied to many other tasks, e.g., pose-guided person image generation [8], [26], [9] and hand gesture-to-gesture translation [10]. However, all of these models require paired training data, which are usually costly to obtain.…”

Section: Related Workmentioning

confidence: 99%

Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation

Tang

Sebe

et al. 2019

2019 International Joint Conference on Neural Networks (IJCNN)

133

View full text Add to dashboard Cite

State-of-the-art models for unpaired image-to-image translation with Generative Adversarial Networks (GANs) can learn the mapping from the source domain to the target domain using a cycle-consistency loss. The intuition behind these models is that if we translate from one domain to the other and back again we should arrive at where we started. However, existing methods always adopt a symmetric network architecture to learn both forward and backward cycles. Because of the task complexity and cycle input difference between the source and target image domains, the inequality in bidirectional forward-backward cycle translations is significant and the amount of information between two domains is different. In this paper, we analyze the limitation of the existing symmetric GAN models in asymmetric translation tasks, and propose an AsymmetricGAN model with both translation and reconstruction generators of unequal sizes and different parameter-sharing strategy to adapt to the asymmetric need in both unsupervised and supervised image-to-image translation tasks. Moreover, the training stage of existing methods has the common problem of model collapse that degrades the quality of the generated images, thus we explore different optimization losses for better training of AsymmetricGAN, and thus make image-to-image translation with higher consistency and better stability. Extensive experiments on both supervised and unsupervised generative tasks with several publicly available datasets demonstrate that the proposed AsymmetricGAN achieves superior model capacity and better generation performance compared with existing GAN models. To the best of our knowledge, we are the first to investigate the asymmetric GAN framework on both unsupervised and supervised image-to-image translation tasks. The source code, data and trained models are available at https://github.com/Ha0Tang/AsymmetricGAN.

show abstract

“…Human Motion Imitation. Recently, most methods are based on conditioned generative adversarial networks (CGAN) [1,3,19,20,22,30] or Variational Auto-Encoder [5]. Their key technical idea is to combine target Figure 3.…”

Section: Related Workmentioning

confidence: 99%

Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis

Liu

Piao

Min

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

287

263

View full text Add to dashboard Cite

We tackle the human motion imitation, appearance transfer, and novel view synthesis within a unified framework, which means that the model once being trained can be used to handle all these tasks. The existing taskspecific methods mainly use 2D keypoints (pose) to estimate the human body structure. However, they only expresses the position information with no abilities to characterize the personalized shape of the individual person and model the limbs rotations. In this paper, we propose to use a 3D body mesh recovery module to disentangle the pose and shape, which can not only model the joint location and rotation but also characterize the personalized body shape. To preserve the source information, such as texture, style, color, and face identity, we propose a Liquid Warping GAN with Liquid Warping Block (LWB) that propagates the source information in both image and feature spaces, and synthesizes an image with respect to the reference. Specifically, the source features are extracted by a denoising convolutional auto-encoder for characterizing the source identity well. Furthermore, our proposed method is able to support a more flexible warping from multiple sources. In addition, we build a new dataset, namely Impersonator (iPER) dataset, for the evaluation of human motion imitation, appearance transfer, and novel view synthesis. Extensive experiments demonstrate the effectiveness of our method in several aspects, such as robustness in occlusion case and preserving face identity, shape consistency and clothes details. All codes and datasets are available on https://svip-lab.github.io/ project/impersonator.html.

show abstract

Disentangled Person Image Generation

Cited by 710 publications

References 30 publications

FaceLift: a transparent deep learning framework to beautify urban scenes

FaceLift: a transparent deep learning framework to beautify urban scenes

Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation

Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis

Contact Info

Product

Resources

About