Analysis and synthesis of facial expressions in knowledge-based coding of facial image sequences

Abstract

This paper proposes a method which analyzes and synthesizes image sequences. and updates a three-dimensional facial model in knowledge-based image coding. Rules for synthesizing the output images are formulated to simulate the facial muscular actions. The input image analysis technique estimates the head motion and the facial actions direclly and robustly without any correspondences. This technique also provides good reproduction of the original images because it is incorporated with the synthesis rules. The presented method for updating the facial model improves analysis accuracy and quality of the synthesized images.

Analysis and synthesis of facial image sequences using physical and anatomical models

“…Realtime synthesis is desirable for model-based analysis-synthesis coding of facial images (see e.g., [ l ] and [lo]). Our approach is philosophically similar to that described in [ 11 and in [4] but differs in the details of the image analysis and resynthesis. In particular, we employ physical rather than geometric modeling methods.…”

“…. , 1v (4) where ya is the coefficient of velocity-proportional damping dissipating kinetic energy in the lattice, g, is the net spring force (3), q, is the net volume restoration force, and f, is the net external force acting on node 2 . It is possible for facial muscle fibers to displace specific attachment nodes by applying driving forces f, to them.…”

<title>Robust tracking of head and arm for model-based image coding</title>

“…head rotation, articulated arm movements) [5], and iii) analysis of nonrigid niotion ( i.e. facial expressions) [5,14]. Often these three tasks are interwoven: the analysis of nonrigid motion requires accurate object tracking, whose accuracy in turn depends on the success of shape reconstruction.…”

“…Often these three tasks are interwoven: the analysis of nonrigid motion requires accurate object tracking, whose accuracy in turn depends on the success of shape reconstruction. As an extension to Choi's analysis-synthesis method [5], we develop a Kalman filtering-based technique to robustly recover 3-D structure and kinematics of the head and arm in view from optical flow. The goal is to improve the system robustness by changing the 3-D wirefraine model adaptively according to recovered imaging geometry and object motion.…”

“…wirefraine fitting) [16], ii) object tracking (i.e. head rotation, articulated arm movements) [5], and iii) analysis of nonrigid niotion ( i.e. facial expressions) [5,14].…”