One-shot Face Reenactment Using Appearance Adaptive Normalization

Face Reenactment with Generative Landmark Guidance

“…GAN feature matching loss [31] proposed by the authors of Vid2Vid is another great contribution to face reenactment. This loss can be seen in [15,19], and [15] is the present state-of-the-art face reenactment method. GAN feature matching loss forces the synthesized images' features in the discriminator to be identical to their corresponding groundtruths images, providing a more direct feedback compared to the min-max loss in Equation 2.1.…”

“…Recent works [2,[15][16][17][18][19][20] propose one-shot or few-shot face reenactment and utilise optical flow to map pixels from the source image to the reenacted image, image warping then becomes an essential operation for these methods. Image warping on convolutional neural networks (CNN) was first proposed in [21], where the model can estimate an optical flow that warps skewed numerical digit back to the regular view, thus improving the classification accuracy.…”

“…When the identity of the source image and that of the driving are different from each other, models trained by self-supervised learning tend to combine the identity features of both identities, the generated face will fail to preserve the identity of the source image. To remedy this issue, the work of [15,16,19] resorts to facial landmark coordinates as a cue to help the model retain the source's identity. In contrast, Monkey-Net [18] and the FirstOrder [17] method focus on animating arbitrary images in stead of face reenactment, these methods does not explicitly leverage facial landmark points, they detect key points in input images and then derive the optical flow based on the motion of key points.…”

“…As discussed in Chapter 2.1, the general idea of image warping based methods [2,[15][16][17][18][19] is to estimate an optical flow based on the source and the driving image, then the source image (or its feature maps) is warped to generate the reenacted face. For instance, the authors of [17,18] estimated key points from input images, their model would then predict an optical flow based these key points to warp the feature maps of input images.…”

DPE: Disentanglement of Pose and Expression for General Video Portrait Editing

“…Recently, most methods [14,25,33,35] endeavor to drive a still portrait image with a video from different perspectives, i.e., one-shot talking face generation. But only a few [18,20,29] make effort to reenact the portrait in a video with another talking video, i.e., video portrait editing.…”