Segmented AAMs Improve Person-Indepedent Face Fitting

Abstract

An Active Appearance Model (AAM) is a variable shape and appearance model built from annotated training images. It has been largely used to synthesize or fit face images. Person-independent face AAM fitting is a challenging open issue. For standard AAMs, fitting a face image for an individual which is not in the training set is often limited in accuracy, thereby restricting the range of application. As a first contribution, we show that the limitation mainly comes from the inability of the AAM appearance counterpart to generalize, i.e. to accurately generate previously unseen visual data. As a second contribution, we propose an efficient person-independent face fitting framework based on what we call multi-level segmented AAMs. Each segment encodes a physically meaningful part of the face, such as an eye. A coarse-to-fine fitting strategy with a gradually increasing number of segments is used in order to ensure a large convergence basin. Fitting accuracy is assessed by comparison with manual labelling statistics constructed from multiple data annotations. Experimental results support the claim that standard AAMs are well-adapted to person-specific fitting while segmented AAMs outperform the classical AAMs in a personindependent context in terms of accuracy, and ability to generate new faces.

Probabilistic constrained adaptive local displacement experts

“…Fitting is then posed as a nonlinear optimization problem, minimizing the difference between the model's appearance and the image [6,17,20]. Due to the high dimensionality of facial appearance it is difficult to compactly represent the whole gamut of appearance variations [13,19], which result in poor fitting performance in the general case. This problem has been partially addressed by using a parts-based representation [3,19], where appearance models are learnt independently for a number of regions of the face.…”

Deformable model fitting with a mixture of local experts

“…Objects that are typically modeled in this way include the human face [3,19] and organs in medical image analysis [22,24]. Numerous representations and fitting strategies have been proposed for these objects, most of which can be categorized based on their representations as being either holistic or patch-based.…”

“…As such, these methods have the capacity to attain highly accurate fitting. However, such a representation generalizes poorly when the object of interest exhibits large amounts of variability, such as in the case of the human face under variations in identity, expression, pose and lighting [8,19]. This is due to the high dimensionality of the represented appearance and the typically limited amount of available training data.…”

“…This is due to the high dimensionality of the represented appearance and the typically limited amount of available training data. It has been shown in [1,19] that a parts-based representation can improve the model's representation capacity as it accounts only for local correlations between pixels values.…”

Face Alignment with Part-Based Modeling

“…Some attempts have been made to improve the robustness, and accuracy of these methods [1,5,9,10,13], but the main problem which remains unsolved in these methods is that they need a good initial estimate and are not able to adapt the model to fit a subject when the initial error is high. As an example in face tracking applications, AAM usually fails to converge when there is a sudden large deformation or motion of the subject.…”