Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?

Abdal, Rameen; Qin, Yipeng; Wonka, Peter

doi:10.1109/iccv.2019.00453

Cited by 997 publications

(841 citation statements)

References 24 publications

Supporting

Mentioning

836

Contrasting

Order By: Relevance

“…The recent improvement made in GAN architectures has enabled us to generate a high quality facial images with a resolution of 1024×1024 pixels using StyleGAN [8]. This is achieved by embedding the images into latent space which is further optimized to synthesize the high quality and high resolution image [1]. As illustrated in Figure 1 the morphed images generated using StyleGAN can be observed to be superior in terms of quality, resolution and visual depiction.…”

Section: Morph Using Styleganmentioning

confidence: 99%

“…Figure 3 depicts the block diagram of the proposed framework for the morphed face generation using a StyleGAN architecture [8]. Given the latent code L 1 of the faces, the StyleGAN In this work, we force a strategy to embed the face image into the latent space (W f ), which is inspired by earlier work [1]. This process enables us to synthesize the data-subject-specific morphed face.…”

Section: Morphed Face Generation Using Styleganmentioning

confidence: 99%

See 1 more Smart Citation

Can GAN Generated Morphs Threaten Face Recognition Systems Equally as Landmark Based Morphs? - Vulnerability and Detection

Venkatesh

Zhang

Ramachandra

et al. 2020

2020 8th International Workshop on Biometrics and Forensics (IWBF)

View full text Add to dashboard Cite

The primary objective of face morphing is to combine face images of different data subjects (e.g. a malicious actor and an accomplice) to generate a face image that can be equally verified for both contributing data subjects. In this paper, we propose a new framework for generating face morphs using a newer Generative Adversarial Network (GAN)-StyleGAN. In contrast to earlier works, we generate realistic morphs of both high-quality and high resolution of 1024×1024 pixels. With the newly created morphing dataset of 2500 morphed face images, we pose a critical question in this work. (i) Can GAN generated morphs threaten Face Recognition Systems (FRS) equally as Landmark based morphs? Seeking an answer, we benchmark the vulnerability of a Commercial-Off-The-Shelf FRS (COTS) and a deep learning-based FRS (ArcFace). This work also benchmarks the detection approaches for both GAN generated morphs against the landmark based morphs using established Morphing Attack Detection (MAD) schemes.

show abstract

Section: Morph Using Styleganmentioning

confidence: 99%

Section: Morphed Face Generation Using Styleganmentioning

confidence: 99%

Can GAN Generated Morphs Threaten Face Recognition Systems Equally as Landmark Based Morphs? - Vulnerability and Detection

Venkatesh

Zhang

Ramachandra

et al. 2020

2020 8th International Workshop on Biometrics and Forensics (IWBF)

View full text Add to dashboard Cite

show abstract

“…Conv (5,5,32) + ReLU Conv (5,5,16) The same architecture is used for CelebA, LFW and FFHQ datasets. The mask acts independently on the RGB channels, as a multiplication without any bias.…”

Section: Spatial Transformermentioning

confidence: 99%

“…The high variance in projection with just a single observation is due to both an overfit surrogate and the identifiability issues arising from not being able to tell apart the underlying true signal from the corruption quality of reprojection (same as eqn. (5)) and corruption mimicking error, which is the same as the optimization cost defined in equation (1). Ideally, we want both to be as small as possible.…”

Section: Properties Of Mimicganmentioning

confidence: 99%

“…shifts like simple corruptions, missing pixels, translations, rotation or scale changes etc. Inferencing in the latent space of GANs has gained a lot of attention recently [1,5,2] with Figure 1: Demonstration of the alternating optimization process. Though initialized randomly, the corruption-mimicking network effectively guides the search in the latent space of the generator to produce an estimate of the true image.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking

2020

View full text Add to dashboard Cite

In the past few years, Generative Adversarial Networks (GANs) have dramatically advanced our ability to represent and parameterize high-dimensional, non-linear image manifolds. As a result, they have been widely adopted across a variety of applications, ranging from challenging inverse problems like image completion, to problems such as anomaly detection and adversarial defense. A recurring theme in many of these applications is the notion of projecting an image observation onto the manifold that is inferred by the generator. In this context, Projected Gradient Descent (PGD) has been the most popular approach, which essentially optimizes for a latent vector that minimizes the discrepancy between a generated image and the given observation. However, PGD is a brittle optimization technique that fails to identify the right projection (or latent vector) when the observation is corrupted, or perturbed even by a small amount. Such corruptions are common in the real world, for example images in the wild come with unknown crops, rotations, missing pixels, or other kinds of non-linear distributional shifts which break current encoding methods, rendering downstream applications unusable. To address this, we propose corruption mimicking -a new robust projection technique, that utilizes a surrogate network to approximate the unknown corruption directly at test time, without the need for additional supervision or data augmentation. The proposed method is significantly more robust than PGD and other competing methods under a wide variety of corruptions, thereby enabling a more effective use of GANs in real-world applications. More importantly, we show that our approach produces state-of-the-art performance in several GAN-based applications -anomaly detection, domain adaptation, and adversarial defense, that benefit from an accurate projection.

show abstract

Facial expression animation through action units transfer in latent space

Tian

Tan

et al. 2020

Computer Animation & Virtual

View full text Add to dashboard Cite

Automatic animation synthesis has attracted much attention from the community. As most existing methods take a small number of discrete expressions rather than continuous expressions, their integrity and reality of the facial expressions is often compromised. In addition, the easy manipulation with simple inputs and unsupervised processing, although being important to the automatic facial expression animation applications, is relatively less concerned. To address these issues, we propose an unsupervised continuous automatic facial expression animation approach through action units (AU) transfer in the latent space of generative adversarial networks. The expression descriptor which is depicted with AU vector is transferred into the input image without the need of labeled pairs of images and even without their expressions and further network training. We also propose a new approach to quickly generate input image's latent code and cluster the boundaries of different AU attributes with their latent codes. Two latent code operators, vector addition and continuous interpolation, are leveraged for facial expression animation simulating align with the boundaries in the latent space. Experiments have shown that the proposed approach is effective on facial expression translation and animation synthesis.

show abstract

Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?

Cited by 997 publications

References 24 publications

Can GAN Generated Morphs Threaten Face Recognition Systems Equally as Landmark Based Morphs? - Vulnerability and Detection

Can GAN Generated Morphs Threaten Face Recognition Systems Equally as Landmark Based Morphs? - Vulnerability and Detection

MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking

Facial expression animation through action units transfer in latent space

Contact Info

Product

Resources

About