Modeling Facial Geometry Using Compositional VAEs

Abstract

We propose a method for learning non-linear face geometry representations using deep generative models. Our model is a variational autoencoder with multiple levels of hidden variables where lower layers capture global geometry and higher ones encode more local deformations. Based on that, we propose a new parameterization of facial geometry that naturally decomposes the structure of the human face into a set of semantically meaningful levels of detail. This parameterization enables us to do model fitting while capturing varying level of detail under different types of geometrical constraints.

Disentangled Representation Learning for 3D Face Shape

“…Tran [38] put forward an encoder-decoder structure for 3D face shape, which is a part of the nonlinear form of 3DMM. Bagautdinov et al [3] propose a compositional Variational Autoencoder structure for representing geometry details in different levels. Tewari et al [3] generate 3D face by self-supervised approach.…”

“…Bagautdinov et al [3] propose a compositional Variational Autoencoder structure for representing geometry details in different levels. Tewari et al [3] generate 3D face by self-supervised approach. Anurag et al [31] propose a graph-based convolutional autoencoder for 3D face shape.…”

“…Inspired by rapid advances of deep learning techniques, learning-based approaches have been proposed to embed 3D face shape into nonlinear parameter spaces, and the representation ability of these methods gets greatly improved, e.g., being able to represent geometry details [3], or reconstructing whole face shapes using very few parameters [31]. However, all of these methods encode the entire face shape into one vector in the latent space, and thus cannot distinguish the identity and expression separately.…”

Towards High-Fidelity Nonlinear 3D Face Morphable Model

“…Our approach offers a balance between global and local models by using a dual-pathway network architecture. Bagautdinov et al [3] try to achieve a similar objective with compositional VAE by introducing multiple layers of hidden variables, but at a cost of extremely large numbers of hidden variables. Residual learning.…”

3D Face Modeling From Diverse Raw Scan Data

“…SfSNet [57] learns shape, albedo and lighting decomposition of a face, from 2D images, instead of 3D scans. Bagautdinov et al [4] learn nonlinear face geometry representations directly from UV maps via a VAE. Ranjan et al [51] introduce a convo- lutional mesh autoencoder to learn nonlinear variations in shape and expression.…”