Facial 3D Shape Estimation from Images for Visual Speech Animation

Musti, Utpala; Zhou, Ziheng; Pietikäinen, Matti

doi:10.1109/icpr.2014.17

Cited by 3 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The transformation matrix in step two can be defined by Theorem 2 as in formula (12). In this formula, = , × ,1 =…”

Section: Theorem 2 the Homogeneous Coordinate Transformationmentioning

confidence: 99%

“…Parke [10] was the first to propose the concept establishing facial animation. Subsequently, many related studies and techniques, including Head Pose Estimation [11], Facial 3D Shape Estimation [12], Head Shop [13], The Digital Emily Project [14], Automatic Generation [15], Kinect-Based Facial Animation [16], Real-Time Facial Animation [17][18][19][20][21], and 3D Facial Similarity Measure [22], emerged.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Improved Surface Simplification Method for Facial Expression Animation Based on Homogeneous Coordinate Transformation Matrix and Maximum Shape Operator

Tseng

2016

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Facial animation is one of the most popular 3D animation topics researched in recent years. However, when using facial animation, a 3D facial animation model has to be stored. This 3D facial animation model requires many triangles to accurately describe and demonstrate facial expression animation because the face often presents a number of different expressions. Consequently, the costs associated with facial animation have increased rapidly. In an effort to reduce storage costs, researchers have sought to simplify 3D animation models using techniques such as Deformation Sensitive Decimation and Feature Edge Quadric. The studies conducted have examined the problems in the homogeneity of the local coordinate system between different expression models and in the retainment of simplified model characteristics. This paper proposes a method that applies Homogeneous Coordinate Transformation Matrix to solve the problem of homogeneity of the local coordinate system and Maximum Shape Operator to detect shape changes in facial animation so as to properly preserve the features of facial expressions. Further, root mean square error and perceived quality error are used to compare the errors generated by different simplification methods in experiments. Experimental results show that, compared with Deformation Sensitive Decimation and Feature Edge Quadric, our method can not only reduce the errors caused by simplification of facial animation, but also retain more facial features.

show abstract

“…The transformation matrix in step two can be defined by Theorem 2 as in formula (12). In this formula, = , × ,1 =…”

Section: Theorem 2 the Homogeneous Coordinate Transformationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Improved Surface Simplification Method for Facial Expression Animation Based on Homogeneous Coordinate Transformation Matrix and Maximum Shape Operator

Tseng

2016

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…The appearance of a visual speech animation system is determined by the underlying face/mouth model, while generating animated talking heads that look like real people is challenging. The existing approaches to talking heads use either imagebased 2D models (Seidlhofer 2009;Zhang et al 2009) or geometry-based 3D ones (Musti et al 2014).Cartoon avatars are relatively easier to build. The more humanlike, realistic avatars, which can be seen in some games or movies, are much harder to build.…”

Section: Face/mouth Modelmentioning

confidence: 99%

Visual Speech Animation

Xie¹,

Wang²,

Yang³

2016

Handbook of Human Motion

View full text Add to dashboard Cite

Visual speech animation (VSA) has many potential applications in humancomputer interaction, assisted language learning, entertainments, and other areas. But it is one of the most challenging tasks in human motion animation because of the complex mechanisms of speech production and facial motion. This chapter surveys the basic principles, state-of-the-art technologies, and featured applications in this area. Specifically, after introducing the basic concepts and the building blocks of a typical VSA system, we showcase a state-of-the-art approach based on the deep bidirectional long short-term memory (DBLSM) recurrent neural networks (RNN) for audio-to-visual mapping, which aims to create a video-realistic talking head. Finally, the Engkoo project from Microsoft is highlighted as a practical application of visual speech animation in language learning.

show abstract