Face Recognition Across Poses Using a Single 3D Reference Model

Abstract

SUMMARYFace recognition across pose is generally tackled by either 2D based or 3D based approaches. The 2D-based often require a training set from which the cross-pose multi-view relationship can be learned and applied for recognition. The 3D based are mostly composed of 3D surface reconstruction of each gallery face, synthesis of 2D images of novel views using the reconstructed model, and match of the synthesized images to the probes. The depth information provides crucial information for arbitrary poses but more methods are yet to be developed. Extended from a latest face reconstruction method using a single 3D reference model and a frontal registered face, this study focuses on using the reconstructed 3D face for recognition. The recognition performance varies with poses, the closer to the front, the better. Several ways to improve the performance are attempted, including different numbers of fiducial points for alignment, multiple reference models considered in the reconstruction phase, and both frontal and profile poses available in the gallery. These attempts make this approach competitive to the state-of-the-art methods.

Benchmarking 3D Pose Estimation for Face Recognition

“…By summing up the square of each equation, 3D-2D pose estimation was transformed into an unconstrained minimization problem (5), which was then solved using the Grobner Basis (GB) Solver: …”

“…However, as pose variations are intrinsically caused by rigid face motion, using 3D data could be beneficial. In addition, using 3D data can also help handle more general and continuous view changes, illumination variance, and unexpected occlusions, examples of which may not be included in the gallery [5,6].…”

“…Although 3DMM can be used to estimate the pose and illumination through shape and texture decomposition, it requires detailed and accurate point-wise correspondence between the 2D image and 3D model as well as (time consuming) parameter tuning. Another promising solution is fitting a general 3D model (reference model) to the 2D face image [5,9] or the 3D face model (if available) [10] of each subject in the gallery to obtain both the fitted 3D face model and the aligned 2D face image during enrollment. In the authentication phase, the fitted 3D model is rotated and translated to fit the 2D probe image and lift a reprojected probe image so that corresponding features between enrollment and authentication are compared in the same normalized 2D space [10].…”

“…As the projection of 2D probe face is determined solely by this projection matrix, the features extracted from the reprojected 2D image are extremely sensitive to pose estimation perturbations. However, among previous work related to MaFR [5,10], almost all of them circumvent the problem by assuming that the landmarks are correct or annotated manually. With perfect landmarks, almost all pose estimation algorithms could easily reach 100% accuracy.…”