3D Face Reconstruction From A Single Image Assisted by 2D Face Images in the Wild

Tu, Xiaoguang; Zhao, Jian; Xie, Mei; Jiang, Zihang; Balamurugan, Akshaya; Luo, Yiyong; Zhao, Yang; He, Lingxiao; Ma, Zheng; Feng, Jiashi

doi:10.1109/tmm.2020.2993962

Cited by 95 publications

(64 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 1 shows the results of the comparative experiment. We directly cite the comparative experimental results from 2DASL [58]. Our method performed well on both datasets, where the AFLW-LFPA results showed 0.1 lower than the average NME of 2DASL.…”

Section: Resultsmentioning

confidence: 98%

FRGAN: A Blind Face Restoration with Generative Adversarial Networks

Wei

Li²,

Chen³

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Recent works based on deep learning and facial priors have performed well in superresolving severely degraded facial images. However, due to the limitation of illumination, pixels of the monitoring probe itself, focusing area, and human motion, the face image is usually blurred or even deformed. To address this problem, we properly propose Face Restoration Generative Adversarial Networks to improve the resolution and restore the details of the blurred face. They include the Head Pose Estimation Network, Postural Transformer Network, and Face Generative Adversarial Networks. In this paper, we employ the following: (i) Swish-B activation function that is used in Face Generative Adversarial Networks to accelerate the convergence speed of the cross-entropy cost function, (ii) a special prejudgment monitor that is added to improve the accuracy of the discriminant, and (iii) the modified Postural Transformer Network that is used with 3D face reconstruction network to correct faces at different expression pose angles. Our method improves the resolution of face image and performs well in image restoration. We demonstrate how our method can produce high-quality faces, and it is superior to the most advanced methods on the reconstruction task of blind faces for in-the-wild images; especially, our 8 × SR SSIM and PSNR are, respectively, 0.078 and 1.16 higher than FSRNet in AFLW.

show abstract

Section: Resultsmentioning

confidence: 98%

FRGAN: A Blind Face Restoration with Generative Adversarial Networks

Wei

Li²,

Chen³

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…3D Face Fitting and Sparse Texture Mapping 1) 3D Face Fitting: 3D face fitting is to estimate the 3D face shape, expression and camera projection matrix from the input 2D face image. Following [26], we estimate the 3D face from a single face image using the 3D Morphable Model (3DMM) [25], [26]. The 3D vertices for describing a 3D face are estimated from a 2D face and expressed as a linear combination over a set of bases:…”

Section: Proposed Methodsmentioning

confidence: 99%

“…3) Loss Function: Given a source face image, we use the 2DAL [26] to regress its 3DMM coefficients, represented as the initial 3D dynamic d 0 . Our prediction model then observes d 0 and proceeds to generate a coefficient sequenced 1:T .…”

Section: B 3d Dynamic Predictionmentioning

confidence: 99%

Image-to-Video Generation via 3D Facial Dynamics

Zou

Zhao³

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

We present a versatile model, FaceAnime, for various video generation tasks from still images. Video generation from a single face image is an interesting problem and usually tackled by utilizing Generative Adversarial Networks (GANs) to integrate information from the input face image and a sequence of sparse facial landmarks. However, the generated face images usually suffer from quality loss, image distortion, identity change, and expression mismatching due to the weak representation capacity of the facial landmarks. In this paper, we propose to "imagine" a face video from a single face image according to the reconstructed 3D face dynamics, aiming to generate a realistic and identity-preserving face video, with precisely predicted pose and facial expression. The 3D dynamics reveal changes of the facial expression and motion, and can serve as a strong prior knowledge for guiding highly realistic face video generation. In particular, we explore face video prediction and exploit a well-designed 3D dynamic prediction network to predict a 3D dynamic sequence for a single face image. The 3D dynamics are then further rendered by the sparse texture mapping algorithm to recover structural details and sparse textures for generating face frames. Our model is versatile for various AR/VR and entertainment applications, such as face video retargeting and face video prediction. Superior experimental results have well demonstrated its effectiveness in generating high-fidelity, identitypreserving, and visually pleasant face video clips from a single source face image.

show abstract

“…Tu et al [ 10 ] developed a 2D assisted self-supervised learning (2DASL) technique for 2D face images. The noisy information of landmarks was used to improve the quality of 3D face models.…”

Section: D Face Reconstruction Techniquesmentioning

confidence: 99%

“…Face reconstruction involves completing the occluded face image. Most 3D face reconstruction techniques use 2D images during the reconstruction process [ 10 – 12 ]. Recently, researchers have started working on mesh and voxel images [ 2 , 8 ].…”

Section: Introductionmentioning

confidence: 99%

3D Face Reconstruction in Deep Learning Era: A Survey

Sharma

Kumar

2022

Arch Computat Methods Eng

View full text Add to dashboard Cite

3D face reconstruction is the most captivating topic in biometrics with the advent of deep learning and readily available graphical processing units. This paper explores the various aspects of 3D face reconstruction techniques. Five techniques have been discussed, namely, deep learning, epipolar geometry, one-shot learning, 3D morphable model, and shape from shading methods. This paper provides an in-depth analysis of 3D face reconstruction using deep learning techniques. The performance analysis of different face reconstruction techniques has been discussed in terms of software, hardware, pros and cons. The challenges and future scope of 3d face reconstruction techniques have also been discussed.

show abstract

3D Face Reconstruction From A Single Image Assisted by 2D Face Images in the Wild

Cited by 95 publications

References 70 publications

FRGAN: A Blind Face Restoration with Generative Adversarial Networks

FRGAN: A Blind Face Restoration with Generative Adversarial Networks

Image-to-Video Generation via 3D Facial Dynamics

3D Face Reconstruction in Deep Learning Era: A Survey

Contact Info

Product

Resources

About