Robust parameterized component analysis: theory and applications to 2D facial appearance models

Abstract

Abstract. Principal Component Analysis (PCA) has been successfully applied to construct linear models of shape, graylevel, and motion. In particular, PCA has been widely used to model the variation in the appearance of people's faces. We extend previous work on facial modeling for tracking faces in video sequences as they undergo significant changes due to facial expressions. Here we develop person-specific facial appearance models (PSFAM), which use modular PCA to model complex intra-person appearance changes. Such models require aligned visual training data; in previous work, this has involved a time consuming and errorprone hand alignment and cropping process. Instead, we introduce parameterized component analysis to learn a subspace that is invariant to affine (or higher order) geometric transformations. The automatic learning of a PSFAM given a training image sequence is posed as a continuous optimization problem and is solved with a mixture of stochastic and deterministic techniques achieving sub-pixel accuracy. We illustrate the use of the 2D PSFAM model with several applications including video-conferencing, realistic avatar animation and eye tracking.

Subspace Procrustes Analysis

“…The computational cost of this method grows polynomially with the number of possible spatial transformations and it can be computationally intensive when working with high-dimensional motion models. To improve upon that, De la Torre and Black [20] proposed parameterized component analysis: a method that learns a subspace of appearance invariant to affine transformations. Miller et al proposed the congealing method [13], which uses an entropy measure to align images with respect to the distribution of the data.…”

“…In computer vision, Procrustes Analysis (PA) has been used extensively to align shapes (e.g., [19,4]) and appearance (e.g., [20,13]) as a pre-processing step to build 2-D models of shape variation. Usually, shape models are learned from a discrete set of 2-D landmarks through a two-step process [8].…”

Joint Unsupervised Face Alignment and Behaviour Analysis

“…We further show how a motion model can be incorporated in ARCA. Our methodology has been motivated by the success of joint alignment and low-rank matrix recovery in person specific scenarios [18,19,20] as well as previous works on parametrized component analysis [21,22]. But our method is radically different to [18], since (1) it extracts latent features rather than image reconstructions, (2) it incorporates a non-rigid motion model guided by a shape model rather than rigid motion used in [18] 1 and (3) it incorporates time dependencies.…”

Performance driven facial animation using illumination independent appearance-based tracking

“…The third approach is based on using linear subspace representations of facial appearance. These representations are gaining popularity, since there are various procedures for automatically learning linear [6,9] and non-linear [10] subspace models and for probabilistically representing the dynamics of appearance variation [15,8]. The major limitation for using appearance-based techniques for facial animation is the impossibility of separating some of the sources of appearance variation, for example facial deformation from illumination.…”