Markerless reconstruction of dynamic facial expressions

Sibbing, Dominik; Habbecke, Martin; Kobbelt, Leif

doi:10.1109/iccvw.2009.5457498

Cited by 5 publications

(6 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It performs a GPU‐assisted stereo reconstruction for a set of camera pairs (for our four camera rig, we can define up to six of such pairs) followed by outlier filtering and disparity refinement and smoothing techniques. The main advantage of this technique is its performance: While the system used in [SHK09, SHK11] is only able to reconstruct 1600 points per second, this GPU‐assisted method reconstructs nearly 300 000 points per second. Most stereo reconstruction techniques heavily depend on the initialization of the depth values.…”

Section: Database Of Dynamic Facial Expressionsmentioning

confidence: 99%

“…In order to be able to efficiently reconstruct the large database of facial expressions and in order to deal with some robustness issues, we designed a new reconstruction system for facial expressions based on the system presented in [SHK09, SHK11]. The first key improvement was the replacement of the Additive Mesh Tracking by the computationally more efficient Inverse Compositional Mesh Tracking , which is described in detail in Section .…”

Section: Database Of Dynamic Facial Expressionsmentioning

confidence: 99%

See 1 more Smart Citation

Building a Large Database of Facial Movements for Deformation Model‐Based 3D Face Tracking

Sibbing

Kobbelt

2017

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

We introduce a new markerless 3D face tracking approach for 2D videos captured by a single consumer grade camera. Our approach takes detected 2D facial features as input and matches them with projections of 3D features of a deformable model to determine its pose and shape. To make the tracking and reconstruction more robust we add a smoothness prior for pose and deformation changes of the faces. Our major contribution lies in the formulation of the deformation prior which we derive from a large database of facial animations showing different (dynamic) facial expressions of a fairly large number of subjects. We split these animation sequences into snippets of fixed length which we use to predict the facial motion based on previous frames. In order to keep the deformation model compact and independent from the individual physiognomy, we represent it by deformation gradients (instead of vertex positions) and apply a principal component analysis in deformation gradient space to extract the major modes of facial deformation. Since the facial deformation is optimized during tracking, it is particularly easy to apply them to other physiognomies and thereby re‐target the facial expressions. We demonstrate the effectiveness of our technique on a number of examples.

show abstract

Section: Database Of Dynamic Facial Expressionsmentioning

confidence: 99%

Section: Database Of Dynamic Facial Expressionsmentioning

confidence: 99%

Building a Large Database of Facial Movements for Deformation Model‐Based 3D Face Tracking

Sibbing

Kobbelt

2017

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

show abstract

“…Many marker-less facial expression analysis methods rely either on tracking points using optical flow [16] or fitting Active Appearance Models [30]. Morphable models in [6], [36], [40] have also been investigated. More recently, 3D data and reconstruction is used to fit directly to the performer [51], [54].…”

Section: Our Applications Of Interest and Relevant Prior Workmentioning

confidence: 99%

Improving facial analysis and performance driven animation through disentangling identity and expression

Rim

Honari

Hasan

et al. 2016

Image and Vision Computing

View full text Add to dashboard Cite

We present techniques for improving performance driven facial animation, emotion recognition, and facial key-point or landmark prediction using learned identity invariant representations. Established approaches to these problems can work well if sufficient examples and labels for a particular identity are available and factors of variation are highly controlled. However, labeled examples of facial expressions, emotions and key-points for new individuals are difficult and costly to obtain. In this paper we improve the ability of techniques to generalize to new and unseen individuals by explicitly modeling previously seen variations related to identity and expression. We use a weakly-supervised approach in which identity labels are used to learn the different factors of variation linked to identity separately from factors related to expression. We show how probabilistic modeling of these sources of variation allows one to learn identity-invariant representations for expressions which can then be used to identity-normalize various procedures for facial expression analysis and animation control. We also show how to extend the widely used techniques of active appearance models and constrained local models through replacing the underlying point distribution models which are typically constructed using principal component analysis with identityexpression factorized representations. We present a wide variety of experiments in which we consistently improve performance on emotion recognition, markerless performance-driven facial animation and facial key-point tracking.

show abstract

“…One of the most prominent methodologies for reconstructing the 3D facial surface from 2D facial images captured in unconstrained environments is the 3D morphable model (3DMM) methodology [10][11][12][13][14][15][16][17] which also constitutes one of the most important recent developments in computational face modelling. Notably, the most well-known publicly available 3DMM is the one presented in [10,11] and has recently been made publicly available in [18].…”

Section: Single Image Reconstructionmentioning

confidence: 99%

Static and dynamic 3D facial expression recognition: A comprehensive survey

Sandbach

Zafeiriou

Pantić

et al. 2012

Image and Vision Computing

328

164

View full text Add to dashboard Cite

a b s t r a c t a r t i c l e i n f oAutomatic facial expression recognition constitutes an active research field due to the latest advances in computing technology that make the user's experience a clear priority. The majority of work conducted in this area involves 2D imagery, despite the problems this presents due to inherent pose and illumination variations. In order to deal with these problems, 3D and 4D (dynamic 3D) recordings are increasingly used in expression analysis research. In this paper we survey the recent advances in 3D and 4D facial expression recognition. We discuss developments in 3D facial data acquisition and tracking, and present currently available 3D/4D face databases suitable for 3D/4D facial expressions analysis as well as the existing facial expression recognition systems that exploit either 3D or 4D data in detail. Finally, challenges that have to be addressed if 3D facial expression recognition systems are to become a part of future applications are extensively discussed.

show abstract

Markerless reconstruction of dynamic facial expressions

Cited by 5 publications

References 34 publications

Building a Large Database of Facial Movements for Deformation Model‐Based 3D Face Tracking

Building a Large Database of Facial Movements for Deformation Model‐Based 3D Face Tracking

Improving facial analysis and performance driven animation through disentangling identity and expression

Static and dynamic 3D facial expression recognition: A comprehensive survey

Contact Info

Product

Resources

About