3D-Aided Deep Pose-Invariant Face Recognition

Abstract

Learning from synthetic faces, though perhaps appealing for high data efficiency, may not bring satisfactory performance due to the distribution discrepancy of the synthetic and real face images. To mitigate this gap, we propose a 3D-Aided Deep Pose-Invariant Face Recognition Model (3D-PIM), which automatically recovers realistic frontal faces from arbitrary poses through a 3D face model in a novel way. Specifically, 3D-PIM incorporates a simulator with the aid of a 3D Morphable Model (3D MM) to obtain shape and appearance prior for accelerating face normalization learning, requiring less training data. It further leverages a global-local Generative Adversarial Network (GAN) with multiple critical improvements as a refiner to enhance the realism of both global structures and local details of the face simulator’s output using unlabelled real data only, while preserving the identity information. Qualitative and quantitative experiments on both controlled and in-the-wild benchmarks clearly demonstrate superiority of the proposed model over state-of-the-arts.

Look across Elapse: Disentangled Representation Learning and Photorealistic Cross-Age Face Synthesis for Age-Invariant Face Recognition

“…Setting-1/Setting-2 HFA [13] 91.14/-CARC [5] 92.80/-MEFA [14] 93.80/-GSM [24] -/94.40 MEFA+SIFT+MLBP [14] 94.59/-LPS+HFA [22] 94.87/-LF-CNN [48] 97.51/-AE-CNN [57] -/98.13 OE-CNN [45] 98. 55 of acquisition. For fair comparisons, Album2 is used for evaluation.…”

Detailed Feature Guided Generative Adversarial Pose Reconstruction Network

“…Zhou et al [12] generated 3D faces based on 3DDFA [13], and then rendered frontal faces. Zhao et al [14] estimated 3D surface based on 3DMM [15], and devised a two pathway structure to deal with global and local texture. These kinds of methods rely on a high precision model trained on a 3D fitting database or a strict and accurate 2D to 3D conversion coordinate system.…”

Multi-View Face Recognition Via Well-Advised Pose Normalization Network

“…Some works incorporate 3DMM into the GAN structure to introduce shape and appearance priors for constraining model training, such as FF-GAN [34], 3D-PIM [35] and HF-PIM [36]. Although the mixture approaches indeed improve the performance under good conditions, they only focus on improving pose normalization network itself, while ignoring the limitation of single-domain learning and the disadvantage of using frontalized faces directly on multiview face recognition.…”

“…When the pose normalization network is optimized after the alternative training between G and D, the face classification network C is reused to extract deep feature embeddings for face recognition. In the general frontalization methods [29], [31], [33]- [35], x is taken the place of x as the input of C, denoted as (x ), for the follow-up identity classification.…”

“…Our method consists of three modules, the generator G, the discriminator D and the classifier C. G and D follow the encoder-decoder structure of DR-GAN by removing the branches of the pose and illumination discriminator, while employing the loss functions of L tex , L sym , L tv , L ip and L domain . C employs a ResNet-50 model [46] pre-trained on the original images of MS-Celeb-1M, it also acts as a baseline [35] in the following experiments.…”