3D landmark‐based face restoration for recognition using variational autoencoder and triplet loss

Abstract

Restoration of a 3D face from the mesh image is highly demanded in computer vision applications. 3D face restoration is a challenging task due to the variation of expression, poses, intrinsic geometries, and textures. The proposed technique consists of two main components, namely face restoration and recognition. A novel three‐dimensional (3D) landmark‐based face restoration method is proposed. 3D facial landmarks are used in the face recognition technique. It uses the principle of reflection and mid‐face plane for the restoration of facial landmarks. By using the restored 3D face, a deep learning‐based face recognition system is developed. It utilizes the concept of deep features from variational autoencoders. Further, these deep feature embeddings are trained using triplet loss training to increase the distance between embeddings of different persons and decreasing the distance between embeddings of the same person. These trained embeddings are used in support vector machine for prediction. The proposed framework is compared with recently developed face recognition techniques in terms of computational time. The proposed technique is able to recognize the person's face with better accuracy than the existing methods. Further, ablation studies are conducted to test the robustness of the proposed technique.

[Retracted] Fast Modelling Algorithm for Realistic Three‐Dimensional Human Face for Film and Television Animation

“…e 3D laser scanner directly obtains face data based on the principle of triangulation. At present, the most famous 3D scanner is developed by Cyberwar, and many research groups use this equipment to conduct research [9,10]. Cai et al [11] used Laplace transform to preprocess the 3D face data collected by the scanner.…”

3D Face Reconstruction in Deep Learning Era: A Survey

“…The occlusion removal is a challenging task for 3D face reconstruction. Researchers are working to handle 3D face occlusion using voxels and 3D landmarks [ 2 , 8 , 9 ]. Sharma and Kumar [ 2 ] developed a voxel-based face reconstruction technique.…”

“…Face alignment may or may not be done for sending it to the reconstruction phase. Sharma and Kumar [ 2 , 8 , 9 ] have not used face alignment for their reconstruction techniques.

Fig.

…”

Masked Face Recognition Using Deep Learning: A Review

“…Then, a decoder reverses the process to generate the key features from the encoded stage with backpropagation at the training time [69]. Autoencoders have been effectively utilized for the task of OFR, such as LSTM-autoencoders [70], double channel SSDA (DC-SSDA) [71], de-corrupt autoencoders [72], and 3D landmark-based variational autoencoder [73].…”

“…MaskTheFace [31] MaskedFace-Net [32], DCNN [33], CYCLE-GAN [34], IAMGAN [35], starGAN [36], segments [39][40][41], regularization [42], sparse rep. [43] Domain-specific models FaceNet [83], SphereFace [8], MFCosface [85], VGGFace [48], DeepID [86], LSTM-autoencoders [70], DC-SSDA [71], de-corrupt autoencoders [72], 3D autoencoder [73], pose invariant FR [77], makeup-invariant [78], DBNs [79,80], attention-aware [82], margin-aware [15] Feature extraction LBPs [44], SIFT [45], HOG [89], codebooks [90], multi-stage mask learning strategy [92], attention-aware and context-aware [93][94][95], GCN [96][97][98] Mask detection R-CNN [101], Fast R-CNN [102], Faster R-CNN [103], context-attention R-CNN [104], FCN [105], U-Net [106], FAN [109], LLE-CNNs [110], ...…”