Hierarchical Image Matching for Pose-invariant Face Recognition

Abstract

The paper addresses the problem of face recognition under arbitrary pose. A hierarchical MRF-based image matching method for finding pixel-wise correspondences between facial images viewed from different angles is proposed and used to densely register a pair of facial images. The goodness-of-match between two faces is then measured in terms of the normalized energy of the match which is a combination of both structural differences between faces as well as their texture distinctiveness. The method needs no training on non-frontal images and circumvents the need for geometrical normalization of facial images. It is also robust to moderate scale changes between images. The proposed approach is evaluated on the CMU PIE database and promising results are obtained.

Warp that smile on your face: Optimal and smooth deformations for face recognition

“…Very similar to our approach is the work of [2], who use tree-based energy minimisation [18] for pose-invariant face recognition with local binary patterns (LBP), although using a model with a less tight lower bound, less strict constraints and no occlusion handling.…”

“…However, several methods exist for example using interior trust regions [24] or data driven iterations [31]. Additionally, maximum aposteriori inference (MAP) in Markov random fields (MRF) is receiving increased attention [19,2,11]. Here, efficient optimisation algorithms like sequential tree-reweighted message passing (TRW-S) [18] allow for finding good optima with a huge number of labels.…”

“…Markov random field (MRF) inference can be used to find a minimum of Eq. (2). Here, we shortly present the tree-reweighted message passing algorithm (TRW-S) by [18], which guarantees convergence to a (local) optimum and gives a lower bound …”

“…In order to obtain robust algorithms with suitable runtimes, approximations and heuristics are introduced by hierarchical approaches [2], splitting of horizontal and vertical displacements which leads to less tight lower bounds, or absolute restrictions of the displacement [23] which implicitly assumes prior knowledge on the variability of the data. Also, it is unclear in what magnitude using an approximation of the global optimum for a given energy function influences the classification performance.…”

“…They can be calculated efficiently using decoupled hidden Markov models (HMMs), but can not find good warpings in the presence of strong nonlinear variations (and can not even cope with rotations). Therefore, stronger warping methods have been proposed [2,19,24,31,11].…”