One Shot Face Swapping on Megapixels

Abstract

Figure 1. Example of a swapped face. Left: source image that represents the identity; Middle: target image that provides the attributes; Right: the swapped face image. All images are in 1024 2 .

Robust Deepfake On Unrestricted Media: Generation And Detection

“…Li et al [4] trained a two-stage framework in a zero-shot learning manner (i.e., self-supervised learning) for high fidelity and occluded face-swapping. Zhu et al [18] extended the latent space to maintain more facial details and then used StyleGAN2 [25] to generate high-resolution swapped facial images. Natsume et al [34] combined separately encoded face and facial landmarks to generate a fake identity.…”

“…In addition, it insightfully discusses potential ways of improving the robustness of deepfake detection methods. Face2Face [5] Deepfakes [2] FaceSwapGAN [3] ZAO [14] Reface [13] Zhu et al [18] FaceApp [12] ProGAN [19] StyleGAN [20] DeepFaceLab [21] Le et al [22] Korshunova et al [23] StarGAN [24] NeuralTextures [6] StyleGANv2 [25] ReenactGAN [26] GANimation [27] StarGANv2 [28] RsGAN [29] FSGAN [7] InterFaceGAN [30] X2Face [31] Zhang et al [32] StyleALAE [33] FSNet [34] Egor et al [11] FaceShifter [4] Zhixin et al [8] ICface [35] Kim et al [36] FaR-GAN [37] The remainder of this chapter is organized as follows. Section 2 introduces deepfake generation methods.…”

High-resolution Face Swapping via Latent Semantics Disentanglement

“…Recently, Naruniec et al [33] swap faces at high resolutions but their model is subject-specific. MegaFS [55] utilizes the prior knowledge of pre-trained StyleGAN [20,21]. It inverts both source and target faces to the latent space, then designs a face transfer block to assemble the latent codes.…”

“…Recently, generative prior has also been introduced for face swapping. Besides MegaFS [55], Nitzan et al [38] transfer attributes from one face to another in the latent space by a fully-connected network. Different from those works, we transfer the target attributes to the source in a more elaborated way.…”

“…Though much progress has been made, many existing GAN-based approaches do not work well on high-resolution faces, due to the compressed representation of end-to-end frameworks [7,26,34], the instability of adversarial training [5], and the limitation in GPU memory size. Recently, Zhu et al [55] utilize the inherent prior knowledge of a pretrained high-resolution GAN model and propose MegaFS for high-resolution face swapping in the latent space of Style-GAN [20,21]. It learns to assemble the inverted latent codes of the source and target images, and directly feed the fused code to a StyleGAN generator to produce the swapped result.…”