Real-time avatar animation with dynamic face texturing

Fechteler, Philipp; Paier, Wolfgang; Hilsmann, Anna; Eisert, Peter

doi:10.1109/icip.2016.7532378

Cited by 11 publications

(12 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Textured animations can be achieved in real‐time [FPHE16, FPE14]. The model is animated with Kinect using retargeting to map the skeleton parameters.…”

Section: Evaluation Discussion and Applicationsmentioning

confidence: 99%

Markerless Multiview Motion Capture with 3D Shape Model Adaptation

Fechteler

Hilsmann

Eisert

2019

Computer Graphics Forum

View full text Add to dashboard Cite

In this paper, we address simultaneous markerless motion and shape capture from 3D input meshes of partial views onto a moving subject. We exploit a computer graphics model based on kinematic skinning as template tracking model. This template model consists of vertices, joints and skinning weights learned a priori from registered full‐body scans, representing true human shape and kinematics‐based shape deformations. Two data‐driven priors are used together with a set of constraints and cues for setting up sufficient correspondences. A Gaussian mixture model‐based pose prior of successive joint configurations is learned to soft‐constrain the attainable pose space to plausible human poses. To make the shape adaptation robust to outliers and non‐visible surface regions and to guide the shape adaptation towards realistically appearing human shapes, we use a mesh‐Laplacian‐based shape prior. Both priors are learned/extracted from the training set of the template model learning phase. The output is a model adapted to the captured subject with respect to shape and kinematic skeleton as well as the animation parameters to resemble the observed movements. With example applications, we demonstrate the benefit of such footage. Experimental evaluations on publicly available datasets show the achieved natural appearance and accuracy.

show abstract

“…Textured animations can be achieved in real‐time [FPHE16, FPE14]. The model is animated with Kinect using retargeting to map the skeleton parameters.…”

Section: Evaluation Discussion and Applicationsmentioning

confidence: 99%

Markerless Multiview Motion Capture with 3D Shape Model Adaptation

Fechteler

Hilsmann

Eisert

2019

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…oral cavity) or simply ignore them. Other works like [Borshukov et al 2006;Carranza et al 2003;Casas et al 2014;Dale et al 2011;Fechteler et al 2014Fechteler et al , 2016Kilner et al 2006;Lipski et al 2011;Paier et al 2015Paier et al , 2017, try to circumvent these limitations by employing image-based rendering approaches, where the facial performance is captured in geometry and texture space. [Dale et al 2011] present a low cost system for facial performance transfer.…”

Section: Related Work 21 Modelling Of Facial Expressionsmentioning

confidence: 99%

“…Using a multilinear face model, they are able to track an actor's facial geometry in 2D video, which enables them to transfer facial performances between different persons via image-based rendering. Similar techniques were used in [Fechteler et al 2014], [Fechteler et al 2016] or [Thies et al 2015]. Another image based animation strategy was proposed in [Casas et al 2014;Paier et al 2015Paier et al , 2017, where low resolution 3D meshes were combined with highly detailed dynamic textures that allow generating high quality renderings.…”

Section: Related Work 21 Modelling Of Facial Expressionsmentioning

confidence: 99%

Neural Face Models for Example-Based Visual Speech Synthesis

Paier

Hilsmann

Eisert

2020

European Conference on Visual Media Production

Self Cite

View full text Add to dashboard Cite

“…oral cavity) or simply ignore them. Other works like [2,6,7,11,15,16,25,29,34,35], try to circumvent these limitations by employing image-based rendering approaches, where the facial performance is captured in geometry and texture space. In [11], Dale et al present a low cost system for facial performance transfer.…”

Section: Related Workmentioning

confidence: 99%

Neural Face Models for Example-Based Visual Speech Synthesis

Paier¹,

Hilsmann²,

Eisert³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Creating realistic animations of human faces with computer graphic models is still a challenging task. It is often solved either with tedious manual work or motion capture based techniques that require specialised and costly hardware. Example based animation approaches circumvent these problems by re-using captured data of real people. This data is split into short motion samples that can be looped or concatenated in order to create novel motion sequences. The obvious advantages of this approach are the simplicity of use and the high realism, since the data exhibits only real deformations. Rather than tuning weights of a complex face rig, the animation task is performed on a higher level by arranging typical motion samples in a way such that the desired facial performance is achieved. Two difficulties with example based approaches, however, are high memory requirements as well as the creation of artefact-free and realistic transitions between motion samples. We solve these problems by combining the realism and simplicity of example-based animations with the advantages of neural face models. Our neural face model is capable of synthesising high quality 3D face geometry and texture according to a compact latent parameter vector. This latent representation reduces memory requirements by a factor of 100 and helps creating seamless transitions between concatenated motion samples. In this paper, we present a marker-less approach for facial motion capture based on multi-view video. Based on the captured data, we learn a neural representation of facial expressions, which is used to seamlessly concatenate facial performances during the animation procedure. We demonstrate the effectiveness of our approach by synthesising mouthings for Swiss-German sign language based on viseme query sequences.

show abstract

Real-time avatar animation with dynamic face texturing

Cited by 11 publications

References 23 publications

Markerless Multiview Motion Capture with 3D Shape Model Adaptation

Markerless Multiview Motion Capture with 3D Shape Model Adaptation

Neural Face Models for Example-Based Visual Speech Synthesis

Neural Face Models for Example-Based Visual Speech Synthesis

Contact Info

Product

Resources

About