Heterogeneous Specular and Diffuse 3-D Surface Approximation for Face Recognition Across Pose

Abstract

Abstract-This paper proposes a novel heterogeneous specular and diffuse (HSD) 3-D surface approximation which considers spatial variability of specular and diffuse reflections in face modelling and recognition. Traditional 3-D face modelling and recognition methods constrain human faces with either the Lambertian assumption or the homogeneity assumption, resulting in suboptimal shape and texture models. The proposed HSD approach allows both specular and diffuse reflectance coefficients to vary spatially to better accommodate surface properties of real human faces. From a small number of face images of a person under different lighting conditions, 3-D shape and surface reflectivity property are estimated using a localized stochastic optimization method. The resultant personalized 3-D face model is used to render novel gallery views under different poses for recognition across pose. The proposed approach is evaluated on both synthetic and real face datasets and benchmarked against the state-of-the-art approaches. Experimental results demonstrated that it can achieve a higher level of performances in modelling accuracy, algorithm reliability, and recognition accuracy, which suggests that face modelling and recognition beyond the Lambertian and homogeneity assumptions is a feasible and better solution towards pose-invariant face recognition.

A Comprehensive Survey on Pose-Invariant Face Recognition

“…However, these results maybe unrealistically optimistic as they assume an ideal situation in which the illumination and expression conditions remain the same across poses. It has been shown that the combined pose and illumination variations have a major influence on the performance of PIFR algorithms on the CMU-PIE database Zhang and Gao (2012); Aldrian and Smith (2013); Kafai et al (2014).…”

Face Recognition Across Poses Using a Single 3D Reference Model

“…Table 2 shows the dimension of WPCA features extracted from the LBP of each pose cluster † . A few best ones from the previous studies were chosen to compare with the state of the art, including TNIP on 24 × 24 intensities (GRAY SRC perform almost equally well as the HSD [8], one of the best 3D methods but requires a few face † The WPCA dimension can be chosen in the training phase for desired performance using the synthesized 2D images in each pose cluster. are similar to each other, the latter is better because it is much faster than the former.…”

“…Approaches for cross-pose recognition can be split into two categories, one is 2D image based [1]- [4] and the other is 3D model based [5]- [8]. More advances have been made on the former which appear to outnumber the latter considerably [9].…”

“…The reconstruction accuracy strongly depends on the correspondences between the fiducial points on the reference faces and those Copyright c 2015 The Institute of Electronics, Information and Communication Engineers on the gallery face, which can be difficult to define precisely. The Heterogeneous Specular and Diffuse (HSD) [8], one of the latest approaches, allows both specular and diffuse reflectance coefficients to vary spatially to better accommodate surface properties of real faces. A few face images under different lighting conditions are needed to estimate the 3-D shape and surface reflectivity using stochastic optimization.…”

“…It is 3D model based in nature, but different from [5]- [8] and others in that it exploits a single 3D reference model and recovers the 3D shape of a 2D face image in the gallery without the need of a dense set of correspondences. It consists of the following steps: (1) 3D reconstruction based on the reference model, (2) model-based synthesis of novel poses, and (3) pose-oriented feature extraction and matching.…”