Automatic Partial Face Alignment in NIR Video Sequences

Abstract

Abstract. Face recognition with partial face images is an important problem in face biometrics. The necessity can arise in not so constrained environments such as in surveillance video, or portal video as provided in Multiple Biometrics Grand Challenge (MBGC). Face alignment with partial face images is a key step toward this challenging problem.In this paper, we present a method for partial face alignment based on scale invariant feature transform (SIFT). We first train a reference model using holistic faces, in which the anchor points and their corresponding descriptor subspaces are learned from initial SIFT keypoints and the relationships between the anchor points are also derived. In the alignment stage, correspondences between the learned holistic face model and an input partial face image are established by matching keypoints of the partial face to the anchor points of the learned face model. Furthermore, shape constraint is used to eliminate outlier correspondences and temporal constraint is explored to find more inliers. Alignment is finally accomplished by solving a similarity transform. Experiments on the MBGC near infrared video sequences show the effectiveness of the proposed method, especially when PCA subspace, shape and temporal constraint are utilized.

Partial face recognition: An alignment free approach

“…For partial faces resulting from a limited field of view, Yang et al [27] and Yi et al [28] proposed an automatic partial face alignment and matching approach. However, their approach requires high resolution images (with an inter-pupil distance of more than 128 pixels) with good skin texture, and it is not applicable to pose variations.…”

Face Alignment under Partial Occlusion in near Infrared Images

“…Our approach outperforms the approach in [10] and is training free. Also good result showed in NIR glass-face alignment and recognition, indicating that the LFPHT is a good way to improve NIR face recognition performances in cases of wearing glasses.…”

“…1. In MBGC-08 Near Infrared partial face video sequences, we compared our approach with Yang's work [10] as well as RANSAC, which is a traditional method and is widely used in parameter estimation as well as pruning local correspondences' outliers [8,10]. In addition to that we compared our approach with eye-detection face alignment method on our self-collected Glass-face face database.…”

“…To deal with the partial occlusion problem, Yang [10] proposed a SIFT [6] based feature codebook learning and RANSAC based parameter estimation algorithm. This approach needs a training process of codebook to represent features in subspace, it performs good in Multiple Biometrics Grand Challenge (MBGC) [9] but has a low detection rate of local features on faces.…”

“…The matching quality of local descriptors is measured by ratio: ri = d1 \ d2, where d1, d2 are respectively the minimum and the second minimum distance. In order to pruning wrong correspondences, we use Shape Context [14] described in [10] to record the relative orientations between correspondences with best matching quality(named "main correspondence") and the other ones both in probe and gallery. Correspondences will be eliminated if the difference of relative orientation with main correspondence are larger than threshold of 30…”