Feature points based facial animation retargeting

Abstract

. Figure 1: The bottom row shows the virtual face animated by retargeting expressions from the source face (top row). AbstractWe present a method for transferring facial animation in real-time. The source animation may be an existing 3D animation or 2D data providing by a video tracker or a motion capture system. Based on two sets of feature points manually selected on the source and target faces (the only manual work required), a RBF network is trained and provides a geometric transformation between the two faces. At each frame, the RBF transformation is applied on the new feature points positions of the source face, resulting in new positions for target feature points according with the expression of the source face and the morphology of the target face. According to their displacements along time, we deform the target mesh on the GPU with the linear blend skinning (LBS) method. In order to make our approach attractive to novice user, we propose a procedural technique to automatically rig the target face by generating vertices weights for the skinning deformation. To summarize, our method provides interactive expression transfer with a minimal human intervention during setup and accepts various kind of animation sources.

2D to 3D space transformation for facial animation based on marker data

“…Motion that occurs in 3D face models is scaled According to the relative scale between the source and the target [4] [6]. Marker points of the source face define the source space while the feature points of the targets face defining the space targets [7] [8].…”

Active nonrigid ICP algorithm

“…Besides, we interpolated new landmarks to avoid irregular deformation. In the existing work of point-driven facial animation, RBF network [24], [25] is trained to map the source points to the template mesh so as to maintain the topology of template. However, in our case, it is beneficial to keep the structure of target face, hence, we only perform rigid alignment instead.…”

Reactive Statistical Mapping: Towards the Sketching of Performative Control with Data

“…Our approach towards bringing together large multimodal datasets and shortterm user interaction is to use machine learning. There is already a very large body of literature on applying machine learning techniques in the fields of gesture or voice recognition [9], gesture or voice synthesis [10], gesture or voice conversion [11], and what is called implicit mapping, i.e. the use of statistical layers as a way of connecting inputs to outputs in interactive systems [12].…”