One-shot Face Reenactment Using Appearance Adaptive Normalization

Abstract

The paper proposes a novel generative adversarial network for one-shot face reenactment, which can animate a single face image to a different pose-and-expression (provided by a driving image) while keeping its original appearance. The core of our network is a novel mechanism called appearance adaptive normalization, which can effectively integrate the appearance information from the input image into our face generator by modulating the feature maps of the generator using the learned adaptive parameters. Furthermore, we specially design a local net to reenact the local facial components (i.e., eyes, nose and mouth) first, which is a much easier task for the network to learn and can in turn provide explicit anchors to guide our face generator to learn the global appearance and pose-and-expression. Extensive quantitative and qualitative experiments demonstrate the significant efficacy of our model compared with prior one-shot methods.

Motion Transformer for Unsupervised Image Animation

“…Image animation: Supervised methods [23,24,11,27,29,55,60,19,5,30,56,36,26] focus on the animation of a specific object type. Among these, the human body [22,29,13,54,31,6,1,33,52,14] and human face [11,12,28,51,46,48,3,41,47,16] are the most popular animation objects. Methods of this kind rely on object-specific landmark detectors, 3D models or other forms of supervision, which are usually pre-trained on a large amount of labeled data.…”