Generating facial expressions of another person based on estimation of muscular contraction parameters from face image

Abstract

SUMMARYWe aim to generate facial expressions of various persons more naturally and without any markers and feature lines. We propose a new approach to estimating muscular contraction parameters for the purpose of generating facial expressions. The muscles of the face are commonly known as the muscles of facial expression. The facial expressions of a human are generated by muscle movements inside the face. If the muscle-based facial model that Waters proposed is used, facial expressions can be generated by muscular contraction parameters. In this paper, first, the facial surface feature points of the face images are detected by imageprocessing methods. Next, the muscular contraction parameters are estimated by the neutral expression and arbitrary expression displacement of the facial model wireframe fitting, based on detected facial surface feature points. Finally, the facial expression is generated by the vertex displacements of an individual facial model based on estimated muscular contraction parameters. Experimental results reveal that our approach can generate various facial expressions of the individual facial model, which corresponds to the facial expression of the actual face image. Additionally, we can generate the facial expressions of another person by using the muscular contraction parameters of that person in an individual facial model.

Synthesis of facial image with expression based on muscular contraction parameters using linear muscle and sphincter muscle

“…4, the proposed model with sphincter muscles supports the generation of expressions of the eyes and mouth, which was impossible in the previous method [13]. The upper and middle rows represent the opening and closing of the eyes and mouth using the vertical model of sphincter muscle, and the lower row shows mouth pursing using the radial model.…”

“…(1) in the previous paper [13] (for the displacement of the vertex group based on the contraction parameter of a linear muscle) when a 2 = 0 and the circumferential influence range of linear muscle is 1. Here r is a parameter that expresses the influence range in the radial direction, and c is a parameter that expresses muscle expansion and contraction.…”

“…The linear muscles considered in this study are the same as in the previous paper [13], and hence explanations are omitted here.…”

“…As explained in the previous paper [13], the facial expressions of the muscle-based facial model are generated by moving vertices P n…”

“…That is, in addition to the previously proposed linear-muscle model [13], we use the newly proposed sphincter-muscle model to approximate the functions of sphincter muscles. Using real images with varied expressions of the eyes and mouth, we verify experimentally that the newly proposed sphincter-muscle model offers better synthesis of facial images with diverse expressions.…”