Efficient generic face model fitting to images and videos

Abstract

In this paper we present a robust and lightweight method for the automatic fitting of deformable 3D face models on facial images. Popular fitting techniques such as those based on statistical models of shape and appearance require a training stage based on a set of facial images and their corresponding facial landmarks, which have to be manually labeled. Therefore, new images in which to fit the model cannot differ too much in shape and appearance (including illumination variation, facial hair, wrinkles, etc) from those used for training. By contrast, our approach can fit a generic face model in two steps: (1) the detection of facial features based on local image gradient analysis and (2) the backprojection of a deformable 3D face model through the optimization of its deformation parameters. The proposed approach can retain the advantages of both learning-free and learning-based approaches. Thus, we can estimate the position, orientation, shape and actions of faces, and initialize user-specific face tracking approaches, such as Online Appearance Models (OAM), which have shown to be more robust * Corresponding author.

Quality-Aware Estimation of Facial Landmarks in Video Sequences

“…According to our observation this is not because of the slip of the proposed method, instead this is because of trivial difference of few pixels between manual annotation from our perception of true landmarks, and the automatic detection by SDM and the proposed method. Figure 7: Comparing the landmark correction results of the proposed system (second and fourth row) against the results of the SDMbased method of [12] (first and third row) in some low quality frames of the Youtube Celebrities database. Table 2 shows the comparison between the Par-CLR based method [13], the SDM-based method [14], and the proposed method in normalized point to point error for all 18 selected video sequences from the Youtube Celebrities database.…”

“…Similar Gauss-Newton or gradient decent based optimization methods for this problem can be found in [10][11]. Standard gradient decent algorithms when applied to AAMs are, however, inefficient in term of computational complexity [2,12]. Two fast AAM fitting approaches were proposed recently in CVPR [13,14].…”

Automated analysis of seizure semiology and brain electrical activity in presurgery evaluation of epilepsy: A focused survey

“…108 Handcrafted features perform inaccurately for the facial expression recognition task under uncontrolled conditions due to a wide range of variations in pose, scale, illumination, and occlusion, and natural variations of individuals in facial shape, texture, and behavior. 108 Handcrafted features perform inaccurately for the facial expression recognition task under uncontrolled conditions due to a wide range of variations in pose, scale, illumination, and occlusion, and natural variations of individuals in facial shape, texture, and behavior.…”

“…Facial motion: The wealth of information present in facial motions has stimulated researchers in applied deep learning architectures to create highly sophisticated models that effectively capture nonlinear mappings of intrinsic features of facial muscle motions. 108 Handcrafted features perform inaccurately for the facial expression recognition task under uncontrolled conditions due to a wide range of variations in pose, scale, illumination, and occlusion, and natural variations of individuals in facial shape, texture, and behavior. Deep learning techniques have proved that they can deal with these challenges effectively 109,110 for different tasks, including recognition of facial semantic features 111,112 and facial motions.…”

Dynamic Adaptation of Personalised Recommendations Based on Tourists’ Affective State

“…Using a two-step, deterministic detection of facial features and adjustment of deformable 3D face model [8], a lightweight and robust method for facial action tracking, a user's facial expression can be estimated on a mid-range mobile device.…”