Spacetime expression cloning for blendshapes

Abstract

The goal of a practical facial animation retargeting system is to reproduce the character of a source animation on a target face while providing room for additional creative control by the animator. This article presents a novel spacetime facial animation retargeting method for blendshape face models. Our approach starts from the basic principle that the source and target movements should be similar. By interpreting movement as the derivative of position with time, and adding suitable boundary conditions, we formulate the retargeting problem as a Poisson equation. Specified (e.g., neutral) expressions at the beginning and end of the animation as well as any user-specified constraints in the middle of the animation serve as boundary conditions. In addition, a model-specific prior is constructed to represent the plausible expression space of the target face during retargeting. A Bayesian formulation is then employed to produce target animation that is consistent with the source movements while satisfying the prior constraints. Since the preservation of temporal derivatives is the primary goal of the optimization, the retargeted motion preserves the rhythm and character of the source movement and is free of temporal jitter. More importantly, our approach provides spacetime editing for the popular blendshape representation of facial models, exhibiting smooth and controlled propagation of user edits across surrounding frames.

Interactive facial expression editing based on spatio-temporal coherency

“…Moreover, we extend our approach by combing expression cloning technique, overcoming the limitation in [17] that only face model having the priori can be edited. Recently, spacetime editing methods [15,[18][19][20][21] that can propagate the editing on a single face frame across the whole sequence have been explored. For example, [19] built a Poisson equation to propagate user's modifications at any frames to the entire sequence.…”

Evaluation of a Facial Animation Authoring Pipeline Seamlessly Supporting Performance Capture and Manual Key-pose Editing

“…None of them maps directly to a control rig, ignoring the need of a post-production editing phase. Seol et al [24] specifically address the problem of providing room for additional editing by an animator. But again, the target of their retargeting method is a set of blendshapes, which, for animators, is harder to use with respect to a control rig.…”

Controllable high-fidelity facial performance transfer

“…Our idea of combining facial retargeting with editing is inspired by recent success on facial expression editing, including both spatial facial editing [Joshi et al 2003;Zhang et al 2004;Lau et al 2007;Meyer and Anderson 2007;Bickel et al 2008;Lau et al 2009;Lewis and Anjyo 2010;Tena et al 2011] and spatial-temporal expression editing [Li and Deng 2008;Seol et al 2012;Akhter et al 2012]. Recent research on spatial facial editing has been focused on utilizing various forms of data-driven models to constrain the solution space of facial editing.…”

“…Recent research on spatial facial editing has been focused on utilizing various forms of data-driven models to constrain the solution space of facial editing. For example, Zhang and col- Our work is most closely related to the work of [Seol et al 2012] because both allow for correction and adjustment of the retargeted motion in the spatial-temporal domain. Our work, however, differs from theirs in several ways.…”