Reconstructing Recognizable 3D Face Shapes based on 3D Morphable Models

Abstract

Many recent works have reconstructed distinctive 3D face shapes by aggregating shape parameters of the same identity and separating those of different people based on parametric models (e.g. 3D morphable models (3DMMs)). However, despite the high accuracy in the face recognition task using these shape parameters, the visual discrimination of face shapes reconstructed from those parameters remains unsatisfactory. Previous works have not answered the following research question: Do discriminative shape parameters guarantee visual discrimination in represented 3D face shapes? This paper analyses the relationship between shape parameters and reconstructed shape geometry, and proposes a novel shape identity-aware regularization (SIR) loss for shape parameters, aiming at increasing discriminability in both the shape parameter and shape geometry domains. Moreover, to cope with the lack of training data containing both landmark and identity annotations, we propose a network structure and an associated training strategy to leverage mixed data containing either identity or landmark labels. In addition, since face recognition accuracy does not mean the recognizability of reconstructed face shapes from the shape parameters, we propose the SIR metric to measure the discriminability of face shapes. We compare our method with existing methods in terms of the reconstruction error, visual discriminability, and face recognition accuracy of the shape parameters and SIR metric. Experimental results show that our method outperforms the state-of-the-art methods. The code will be released at https:// github.com/ a686432/ SIR.

Sphere Face Model: A 3D morphable model with hypersphere manifold latent space using joint 2D/3D training

“…is a linear function and the basis (mentioned later in Section 4) is orthonormal, the above condition can be met (the property is proved in Ref. [42]). Thus we use orthonormal basis in SFM.…”

“…Liu et al [11] and Sanyal et al [9] use a face recognition loss to push away the shape parameters of different people while aggregating those of the same person. Jiang et al [42] propose that simply applying the face recognition loss function to the shape parameter does not guarantee shape consistency. They explore the relationship of shape parameter discrimination and geometric visual discrimination and propose the SIR loss, which increases discriminability in both the shape parameter and shape geometry domain.…”

“…The identity embedding trained with softmaxbased classification is distributed on a hypersphere. Previous works [11,42] impose the softmax-based loss on shape parameters. However, the face parameters constrained by the face recognition loss function will make the face parameters tend to have a hyperspherical distribution.…”

“…Our results are obtained using the weighted center loss. We compare our results with 3DMM-CNN [10], Liu et al [11], D3FR [34], TDDFA [33], MGCNet [72], Jiang et al [42], RingNet [9], and DECA [41]. PCA is the PCA-based model using the same 3D face dataset as SFM.…”