RGB-D Based Face Reconstruction and Recognition

Abstract

Most RGB-D based research focuses on gesture analysis, scene reconstruction and SLAM, but only few study its impacts on face recognition. A common yet challenging scenario considered in face recognition across pose takes a single 2D face of frontal pose as the galley and other poses as the probe set. We consider a similar scenario but with a RGB-D image pair taken at frontal pose in the gallery, only 2D images with a large scope of poses in the probe set, and study the advantage of the additional depth map on top of the regular RGB image. We formulate the 3D face reconstruction using the RGB-D image as a constrained optimization, and compare the results with different reconstruction settings. The reconstructed 3D face allows the generation of 2D face with specific poses, which can be matched with the probes. Experiments on the Biwi Kinect Head Pose Database and Eurecom Database show that the additional depth map substantially improves the crosspose recognition performance, and the depth-based component selection also improves the recognition under occlusion and expression variation.

Discriminant Patch Representation for RGB-D Face Recognition using Convolutional Neural Networks

“…That is, if we take one sample for testing, no corresponding pose exists anymore in the gallery. In contrary and as reported previously in the related work section, (Ciaccio et al, 2013;Hsu et al, 2014;Li et al, 2013) pre-processings allow to tackle this issue as they either augment the gallery by generating new poses, correct the pose or symmetrically filling the self occluded part in the face. Beyond these well engineered pre-processings, in this work we aim to focus on estimating optimal RGB-D data representation for face recognition applications.…”

“…For the first set of experiments, and on CurtinFaces dataset, we report our results as well as those of (Hsu et al, 2014) (Li et al, 2013), (Ciaccio et al, 2013), (Kaashki and Safabakhsh, 2018), and (Grati et al, 2016). To make a fair comparison, we use the same protocol as (Li et al, 2013).…”

“…We note that the most prominent methods (Hsu et al, 2014) (Li et al, 2013) aim principally to overcome the issues of pose variation either with pose correction or with augmenting the gallery through generating new images in different view or even capturing multi-view data for each face in the gallery. Data representation and appropriate feature extraction remains underestimated in the aforementioned works and they settle for hand-crafted features combinations and fusion.…”

Scrutinize Study on Face Recognition by Pose Invariant Methodology

“…In [14] optimum image processing technique has been proposed identifying error rates of face recognition methods. It was also investigated that fisher face methods used for face recognition has not performed as predicted by the researchers.…”

3D Face Reconstruction System Based on Deep Learning and Sparse Face Model

“…GS Hsu et al [2] adopted the additional depth map on top of the regular RGB image and formulated the 3D face reconstruction using the RGB-D image as a constrained optimization. Feng Liu et al [3] computed a series of coarse-to-fine shape adjustments to the initial 3D face shape through cascaded repressor based on the deviations between the input landmarks and the landmarks rendered from the reconstructed 3D faces.…”