The 3D Menpo Facial Landmark Tracking Challenge

Zafeiriou, Stefanos; Chrysos, Grigorios; Roussos, Anastasios; Ververas, Evangelos; Deng, Jiankang; Trigeorgis, George

doi:10.1109/iccvw.2017.16

Cited by 43 publications

(43 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhu et al (2016c) proposed a method to synthesise large-scale training samples in profile views [300W-LP (Zhu et al 2016c)] and employed CNN to fit the dense 3D face model to facial images. In Zafeiriou et al (2017a), an elaborate semi-automatic methodology was proposed to provide highquality 3D landmark annotations for face images and videos. From the results of the Menpo 3D challenge (Zafeiriou et al 2017a), we find stacked hourglass network (Xiong et al 2017) once again set up state-of-the-art performance.…”

Section: Datasetsmentioning

confidence: 99%

The Menpo Benchmark for Multi-pose 2D and 3D Facial Landmark Localisation and Tracking

et al. 2018

Self Cite

View full text Add to dashboard Cite

In this article, we present the Menpo 2D and Menpo 3D benchmarks, two new datasets for multi-pose 2D and 3D facial landmark localisation and tracking. In contrast to the previous benchmarks such as 300W and 300VW, the proposed benchmarks contain facial images in both semi-frontal and profile pose. We introduce an elaborate semi-automatic methodology for providing high-quality annotations for both the Menpo 2D and Menpo 3D benchmarks. In Menpo 2D benchmark, different visible landmark configurations are designed for semi-frontal and profile faces, thus making the 2D face alignment full-pose. In Menpo 3D benchmark, a united landmark configuration is designed for both semi-frontal and profile faces based on the correspondence with a 3D face model, thus making face alignment not only full-pose but also corresponding to the realworld 3D space. Based on the considerable number of annotated images, we organised Menpo 2D Challenge and Menpo 3D Challenge for face alignment under large pose variations in conjunction with CVPR 2017 and ICCV 2017, respectively. The results of these challenges demonstrate that recent deep learning architectures, when trained with the abundant data, lead to excellent results. We also provide a very simple, yet effective solution, named Cascade Multi-view Hourglass Model, to 2D and 3D face alignment. In our method, we take advantage of all 2D and 3D facial landmark annotations in a joint way. We not only capitalise on the correspondences between the semi-frontal and profile 2D facial landmarks but also employ joint supervision from both 2D and 3D facial landmarks. Finally, we discuss future directions on the topic of face alignment. Keywords 2D face alignment • 3D face alignment • Menpo challenge 1 Introduction Facial landmark localisation and tracking on images and videos captured in unconstrained recording conditions is a problem that has received a lot of attention during the past few years. This is attributed to the fact that it is a neces

show abstract

Section: Datasetsmentioning

confidence: 99%

The Menpo Benchmark for Multi-pose 2D and 3D Facial Landmark Localisation and Tracking

et al. 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…The facial shape rigidity assumption throughout the whole video is rather tight. However, as verified experimentally in [47], once provided with a significant number of frames, it provides a very robust initialisation of the camera parameters even in cases of large facial deformation.…”

Section: D Rigid Initialisation Using 3dmmmentioning

confidence: 82%

Combining dense nonrigid structure from motion and 3D morphable models for monocular 4D face reconstruction

Koujan

Roussos

2018

Proceedings of the 15th ACM SIGGRAPH European Conference on Visual Media Production

Self Cite

View full text Add to dashboard Cite

Monocular 4D face reconstruction is a challenging problem, especially in the case that the input video is captured under unconstrained conditions, i.e. "in the wild". The majority of the state-of-the-art approaches build upon 3D Morphable Modelling (3DMM), which has been proven to be more robust than model-free approaches such as Shape from Shading (SfS) or Structure from Motion (SfM). While offering visually plausible shape reconstruction results that resemble real faces, 3DMMs adhere to the model space learned from exemplar faces during the training phase, often yielding facial reconstructions that are excessively smooth and look too similar even across captured faces with completely different facial characteristics. This is due to the fact that 3DMMs are typically used as hard constraints on the reconstructed 3D shape. To overcome these limitations, in this paper we propose to combine 3DMMs with Dense Nonrigid Structure from Motion (DNSM), which is much less robust but has the potential of reconstructing fine details and capturing the subject-specific facial characteristics of every input. We effectively combine the best of both worlds by introducing a novel dense variational framework, which we solve efficiently by designing a convex optimisation strategy. In contrast to previous methods, we incorporate 3DMM as a soft constraint, penalizing both departure of reconstructed faces from the 3DMM subspace and variation of the identity component of the 3DMM over different frames of the input video. As demonstrated in qualitative and quantitative experiments, our method is robust, accurately estimates the 3D facial shape over time and outperforms other state-of-the-art methods of 4D face reconstruction.

show abstract

“…(b) Mean difference in 3d landmark locations. Figure 5: Face tracking results using the proposed method, as evaluated by the "3D Face Tracking in-the-wild Competition" [14] organizers.…”

Section: Resultsmentioning

confidence: 99%

“…As part of the 1st 3D Face Tracking in-the-wild Competition [14], the proposed method was used to estimate sparse landmarks on 30 short video clips. This was accomplished using face detection, 3D model estimation, landmark interpolation, and finally smoothing.…”

Section: D Face Tracking Challengementioning

confidence: 99%

Pix2Face: Direct 3D Face Model Estimation

Crispell

Bazik²

2017

2017 IEEE International Conference on Computer Vision Workshops (ICCVW)

View full text Add to dashboard Cite

An efficient, fully automatic method for 3D face shape and pose estimation in unconstrained 2D imagery is presented. The proposed method jointly estimates a dense set of 3D landmarks and facial geometry using a single pass of a modified version of the popular "U-Net" neural network architecture. Additionally, we propose a method for directly estimating a set of 3D Morphable Model (3DMM) parameters, using the estimated 3D landmarks and geometry as constraints in a simple linear system. Qualitative modeling results are presented, as well as quantitative evaluation of predicted 3D face landmarks in unconstrained video sequences.

show abstract

The 3D Menpo Facial Landmark Tracking Challenge

Abstract: captured "in-the-wild" and (b)

Cited by 43 publications

References 23 publications

The Menpo Benchmark for Multi-pose 2D and 3D Facial Landmark Localisation and Tracking

The Menpo Benchmark for Multi-pose 2D and 3D Facial Landmark Localisation and Tracking

Combining dense nonrigid structure from motion and 3D morphable models for monocular 4D face reconstruction

Pix2Face: Direct 3D Face Model Estimation

Contact Info

Product

Resources

About