Riggable 3D Face Reconstruction via In-Network Optimization

Bai, Ziqian; Cui, Zhaopeng; Liu, Xiaoming; Tan, Ping

doi:10.48550/arxiv.2104.03493

Cited by 2 publications

(2 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dou et al (Dou and Kakadiaris 2018) and Ramon et al (Ramon, Escur, and Giroi Nieto 2019) proposes to recover the face model from multiview images using a subspace representation of the 3D facial shape and a deep recurrent neural network to fuse the identity related features. Bai et al (Bai et al 2020(Bai et al , 2021 propose to optimize the 3D face shape by explicitly enforcing multi-view appearance consistency, which makes it possible to recover shape details according to conventional multiview stereo methods.…”

Section: Related Workmentioning

confidence: 99%

Detailed Facial Geometry Recovery from Multi-View Images by Learning an Implicit Function

Xiao¹,

Zhu²,

Huang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recovering detailed facial geometry from a set of calibrated multi-view images is valuable for its wide range of applications. Traditional multi-view stereo (MVS) methods adopt optimization methods to regularize the matching cost. Recently, learning-based methods integrate all these into an endto-end neural network and show superiority of efficiency. In this paper, we propose a novel architecture to recover extremely detailed 3D faces in roughly 10 seconds. Unlike previous learning-based methods that regularize the cost volume via 3D CNN, we propose to learn an implicit function for regressing the matching cost. By fitting a 3D morphable model from multi-view images, the features of multiple images are extracted and aggregated in the mesh-attached UV space, which makes the implicit function more effective in recovering detailed facial shape. Our method outperforms SOTA learning-based MVS in accuracy by a large margin on the FaceScape dataset. The code and data will be released soon.

show abstract

Section: Related Workmentioning

confidence: 99%

Detailed Facial Geometry Recovery from Multi-View Images by Learning an Implicit Function

Xiao¹,

Zhu²,

Huang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Dou et al (Dou and Kakadiaris 2018) and Ramon et al (Ramon, Escur, and Giroi Nieto 2019) proposes to recover the face model from multiview images using a subspace representation of the 3D facial shape and a deep recurrent neural network to fuse the identity-related features. Bai et al (Bai et al 2020(Bai et al , 2021 propose to optimize the 3D face shape by explicitly enforcing multi-view appearance consistency, which makes it possible to recover shape details according to conventional multi-view stereo methods.…”

Section: Related Workmentioning

confidence: 99%

Detailed Facial Geometry Recovery from Multi-View Images by Learning an Implicit Function

Xiao

Zhu

Yang

et al. 2022

AAAI

View full text Add to dashboard Cite

Recovering detailed facial geometry from a set of calibrated multi-view images is valuable for its wide range of applications. Traditional multi-view stereo (MVS) methods adopt an optimization-based scheme to regularize the matching cost. Recently, learning-based methods integrate all these into an end-to-end neural network and show superiority of efficiency. In this paper, we propose a novel architecture to recover extremely detailed 3D faces within dozens of seconds. Unlike previous learning-based methods that regularize the cost volume via 3D CNN, we propose to learn an implicit function for regressing the matching cost. By fitting a 3D morphable model from multi-view images, the features of multiple images are extracted and aggregated in the mesh-attached UV space, which makes the implicit function more effective in recovering detailed facial shape. Our method outperforms SOTA learning-based MVS in accuracy by a large margin on the FaceScape dataset. The code and data are released in https://github.com/zhuhao-nju/mvfr.

show abstract

Riggable 3D Face Reconstruction via In-Network Optimization

Abstract: Figure 1. Our method estimates personalized face rigs and per-image reconstructions from monocular images with good reconstruction quality and supports video retargeting to different actors. The code is available at https://github.com/zqbai-jeremy/INORig.

Cited by 2 publications

References 52 publications

Detailed Facial Geometry Recovery from Multi-View Images by Learning an Implicit Function

Detailed Facial Geometry Recovery from Multi-View Images by Learning an Implicit Function

Detailed Facial Geometry Recovery from Multi-View Images by Learning an Implicit Function

Contact Info

Product

Resources

About