Analysis and synthesis of facial image sequences using physical and anatomical models

Abstract

Abstrucf-We present a new approach to the analysis of dynamic facial images for the purposes of estimating and resynthesizing dynamic facial expressions. The approach exploits a sophisticated generative model of the human face originally developed for realistic facial animation. The face model, which may be simulated and rendered at interactive rates on a graphics workstation, incorporates a physics-based synthetic facial tissue and a set of anatomically motivated facial muscle actuators. We consider the estimation of dynamic facial muscle contractions from video sequences of expressive human faces. We develop an estimation technique that uses deformable contour models (snakes) to track the nonrigid motions of facial features in video images. The technique estimates muscle actuator controls with sufficient accuracy to permit the face model to resynthesize transient expressions.

Multimodal Human Machine Interactions in Virtual and Augmented Reality

“…Contour of features can be tracked with active contours, or "snakes", that obey internal and external forces. Internal energy terms account for the shape of the feature and smoothness of the contour while the external energy attracts the snake towards feature contours in the image [72]. Face features (eyes, mouth) can be described with sets of wavelet components and linked together in an elastic graph [81].…”

Active blobs: region-based, deformable appearance models

“…One of the problems of fitting a 3D model to a single image is that depth is almost impossible to determine, and the tracking problem is generally underconstrained. As a result, 3D model-based nonrigid motion tracking techniques require either strong prior assumptions about the shape of the object (e.g., symmetry [57], deformable superquadrics [37,40,66], human faces [15,17,56]) or that multiple views of the object are available. By using a 2D deformable image template, the active blobs formulation has the advantage of being able to track general nonrigid motion within the image plane without the need to define a full 3D model nor strong prior assumptions about the shape of the object to be tracked.…”

Performance driven facial animation using illumination independent appearance-based tracking

“…Feature-based procedures were the first tracking algorithms to be introduced. They are based on tracking a discrete set of texture elements such as eye or nose corners and the contours of expressive regions (eyes, eyelids or mouth) [2,14]. They can only estimate the motion of textured regions and, therefore, they provide sparse information about the deformation of the face.…”