Nonlinear 3D Face Morphable Model

Tran, Luan; Liu, Xiaoming

doi:10.1109/cvpr.2018.00767

Cited by 425 publications

(319 citation statements)

References 46 publications

Supporting

Mentioning

317

Contrasting

Unclassified

Order By: Relevance

“…They also train their encoder using synthetic data and its corresponding 3D parameters. Tran and Liu [36] learn a nonlinear 3DMM model by using an analytically differentiable rendering layer and in a weakly supervised fashion with 3D data.…”

Section: Related Workmentioning

confidence: 99%

Learning to Regress 3D Face Shape and Expression From an Image Without 3D Supervision

Sanyal

Bolkart

Feng

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

298

286

View full text Add to dashboard Cite

Figure 1: Without 3D supervision, RingNet learns a mapping from the pixels of a single image to the 3D facial parameters of the FLAME model [21]. Top: Images are from the CelebA dataset [22]. Bottom: estimated shape, pose and expression. AbstractThe estimation of 3D face shape from a single image must be robust to variations in lighting, head pose, expression, facial hair, makeup, and occlusions. Robustness requires a large training set of in-the-wild images, which by construction, lack ground truth 3D shape. To train a network without any 2D-to-3D supervision, we present RingNet, which learns to compute 3D face shape from a single image. Our key observation is that an individual's face shape is constant across images, regardless of expression, pose, lighting, etc. RingNet leverages multiple images of a person and automatically detected 2D face features. It uses a novel loss that encourages the face shape to be similar when the identity is the same and different for different people. We achieve invariance to expression by representing the face using the FLAME model. Once trained, our method takes a single image and outputs the parameters of FLAME, which can be readily animated. Additionally we create a new database of faces "not quite in-the-wild" (NoW) with 3D head scans and high-resolution images of the subjects in a wide variety of conditions. We evaluate publicly available methods and find that RingNet is more accurate than methods that use 3D supervision. The dataset, model, and results are available for research purposes at

show abstract

Section: Related Workmentioning

confidence: 99%

Learning to Regress 3D Face Shape and Expression From an Image Without 3D Supervision

Sanyal

Bolkart

Feng

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

298

286

View full text Add to dashboard Cite

show abstract

“…Besides model-based approaches [43,42,31] representing data with semantic latent vectors; data-driven disentangled representation learning approaches are gaining popularity in computer vision community. DrNet [14] disentangles content and pose vectors with a two-encoders architecture, which removes content information in the pose vector by generative adversarial training.…”

Section: Related Workmentioning

confidence: 99%

Gait Recognition via Disentangled Representation Learning

Zhang

Tran

Yin

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

230

139

View full text Add to dashboard Cite

Gait, the walking pattern of individuals, is one of the most important biometrics modalities. Most of the existing gait recognition methods take silhouettes or articulated body models as the gait features. These methods suffer from degraded recognition performance when handling confounding variables, such as clothing, carrying and view angle. To remedy this issue, we propose a novel AutoEncoder framework to explicitly disentangle pose and appearance features from RGB imagery and the LSTM-based integration of pose features over time produces the gait feature. In addition, we collect a Frontal-View Gait (FVG) dataset to focus on gait recognition from frontal-view walking, which is a challenging problem since it contains minimal gait cues compared to other views. FVG also includes other important variations, e.g., walking speed, carrying, and clothing. With extensive experiments on CASIA-B, USF and FVG datasets, our method demonstrates superior performance to the state of the arts quantitatively, the ability of feature disentanglement qualitatively, and promising computational efficiency.

show abstract

“…Face Manipulation. In the literature of face image manipulation, besides classic mass-spring models and 2D/3D morphing methods [42,43,44], recently, significant progress has been achieved by leveraging the power of Generative Adversarial Networks (GANs) [32,22,9,37,53,51] for photo-realistic synthesis results. To improve the robustness and diversity of GANs, tweaks on various aspects are explored.…”

Section: Related Workmentioning

confidence: 99%

Make a Face: Towards Arbitrary High Fidelity Face Manipulation

Qian

Lin

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

InputFace Manipulation Results of Our Model Figure 1: Face manipulation results on in-the-wild samples via transferring knowledge learned from the CelebA dataset. The first column shows input images and the remainders are images generated by AF-VAE with target expression/rotation boundary maps as the condition. Note that the model is fine-tuned with movie clip frames from YouTube of 256 × 256 resolution. All the generated poses are unseen before. AbstractRecent studies have shown remarkable success in face manipulation task with the advance of GANs and VAEs paradigms, but the outputs are sometimes limited to lowresolution and lack of diversity.In this work, we propose Additive Focal Variational Auto-encoder (AF-VAE), a novel approach that can arbitrarily manipulate high-resolution face images using a simple yet effective model and only weak supervision of reconstruction and KL divergence losses. First, a novel additive Gaussian Mixture assumption is introduced with an unsupervised clustering mechanism in the structural latent 1 Work done during an internship at SenseTime Research. space, which endows better disentanglement and boosts multi-modal representation with external memory. Second, to improve the perceptual quality of synthesized results, two simple strategies in architecture design are further tailored and discussed on the behavior of Human Visual System (HVS) for the first time, allowing for fine control over the model complexity and sample quality. Human opinion studies and new state-of-the-art Inception Score (IS) / Fréchet Inception Distance (FID) demonstrate the superiority of our approach over existing algorithms, advancing both the fidelity and extremity of face manipulation task.

show abstract

Nonlinear 3D Face Morphable Model

Cited by 425 publications

References 46 publications

Learning to Regress 3D Face Shape and Expression From an Image Without 3D Supervision

Learning to Regress 3D Face Shape and Expression From an Image Without 3D Supervision

Gait Recognition via Disentangled Representation Learning

Make a Face: Towards Arbitrary High Fidelity Face Manipulation

Contact Info

Product

Resources

About