Few-Shot Adversarial Learning of Realistic Neural Talking Head Models

Zakharov, Egor; Shysheya, Aliaksandra; Burkov, Egor; Lempitsky, Victor

doi:10.48550/arxiv.1905.08233

Cited by 33 publications

(69 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multi-domain image-to-image translation with cGAN: Conditioning GANs has been an active research field ever since Mirza et al [29] first showed that auxiliary information such as target label can be fed into standard GANs [12] to generate images conditioned on the auxiliary information. Various forms of representations such as one-hot encoded target domain label [6,8,17,25,51], source domain in the form of semantic text or layout representa-tions [37,49,18,52], source information in the form of images [20,19,26], geometry captured in the form of sparse landmarks [48] have successfully been applied to control the nature of synthetic data. Popular image-to-image translation works such as [20,22] generate visually pleasing images, and may use reconstruction loss such as cycle consistency to preserve the major attributes of the source image [54].…”

Section: Related Workmentioning

confidence: 99%

Label Geometry Aware Discriminator for Conditional Generative Networks

Sapkota¹,

Khanal²,

Bhattarai³

et al. 2021

Preprint

View full text Add to dashboard Cite

Multi-domain image-to-image translation with conditional Generative Adversarial Networks (GANs) can generate highly photo realistic images with desired target classes, yet these synthetic images have not always been helpful to improve downstream supervised tasks such as image classification. Improving downstream tasks with synthetic examples requires generating images with high fidelity to the unknown conditional distribution of the target class, which many labeled conditional GANs attempt to achieve by adding soft-max cross-entropy loss based auxiliary classifier in the discriminator. As recent studies suggest that the soft-max loss in Euclidean space of deep feature does not leverage their intrinsic angular distribution, we propose to replace this loss in auxiliary classifier with an additive angular margin (AAM) loss that takes benefit of the intrinsic angular distribution, and promotes intra-class compactness and inter-class separation to help generator synthesize high fidelity images.We validate our method on RaFD and CIFAR-100, two challenging face expression and natural image classification data set. Our method outperforms state-of-the-art methods in several different evaluation criteria including recently proposed GAN-train and GAN-test metrics designed to assess the impact of synthetic data on downstream classification task, assessing the usefulness in data augmentation for supervised tasks with prediction accuracy score and average confidence score, and the well known FID metric.

show abstract

Section: Related Workmentioning

confidence: 99%

Label Geometry Aware Discriminator for Conditional Generative Networks

Sapkota¹,

Khanal²,

Bhattarai³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Perceptual loss [21] between Ĵfake and J. Perceptual loss is significant to preserve the identity information and high-level semantic features of the face images. We follow the work of [45] and employ the pre-trained VGG-Face [31] network to extract the features:…”

Section: The Training Process Of the Pas Is Guided By The Following L...mentioning

confidence: 99%

Pixel Sampling for Style Preserving Face Pose Editing

Yin

Huang

Yang

et al. 2020

2020 IEEE International Joint Conference on Biometrics (IJCB)

View full text Add to dashboard Cite

The existing auto-encoder based face pose editing methods primarily focus on modeling the identity preserving ability during pose synthesis, but are less able to preserve the image style properly, which refers to the color, brightness, saturation, etc. In this paper, we take advantage of the well-known frontal/profile optical illusion and present a novel two-stage approach to solve the aforementioned dilemma, where the task of face pose manipulation is cast into face inpainting. By selectively sampling pixels from the input face and slightly adjust their relative locations with the proposed "Pixel Attention Sampling" module, the face editing result faithfully keeps the identity information as well as the image style unchanged. By leveraging highdimensional embedding at the inpainting stage, finer details are generated. Further, with the 3D facial landmarks as guidance, our method is able to manipulate face pose in three degrees of freedom, i.e., yaw, pitch, and roll, resulting in more flexible face pose editing than merely controlling the yaw angle as usually achieved by the current state-ofthe-art. Both the qualitative and quantitative evaluations validate the superiority of the proposed approach.

show abstract

“…To solve this issue, few-shot or even one-shot face reenactment methods have also been developed in the recent work [29,31,33]. Wiles et al [29] propose a model, namely X2Face, that is able to use facial landmarks or audio to drive the input source image to a target expression.…”

Section: Related Workmentioning

confidence: 99%

“…Then it produces an intermediate interpolation map given the target expression to be used for transferring the frontalized face. Zakharov et al [31] present a few-shot learning approach that achieves the face reenactment given a few, or even one, source images. Unlike the X2Face model, their method is able to directly transfer the expression without the intermediate boundary latent space [30] or interpolation map [29].…”

Section: Related Workmentioning

confidence: 99%

FaR-GAN for One-Shot Face Reenactment

Hao,

Baireddy,

Reibman

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

Few-Shot Adversarial Learning of Realistic Neural Talking Head Models

Cited by 33 publications

References 0 publications

Label Geometry Aware Discriminator for Conditional Generative Networks

Label Geometry Aware Discriminator for Conditional Generative Networks

Pixel Sampling for Style Preserving Face Pose Editing

FaR-GAN for One-Shot Face Reenactment

Contact Info

Product

Resources

About