What Does 2D Geometric Information Really Tell Us About 3D Face Shape?

Abstract

A face image contains geometric cues in the form of configurational information and contours that can be used to estimate 3D face shape. While it is clear that 3D reconstruction from 2D points is highly ambiguous if no further constraints are enforced, one might expect that the face-space constraint solves this problem. We show that this is not the case and that geometric information is an ambiguous cue. There are two sources for this ambiguity. The first is that, within the space of 3D face shapes, there are flexibility modes that remain when some parts of the face are fixed. The second occurs only under perspective projection and is a result of perspective transformation as camera distance varies. Two different faces, when viewed at different distances, can give rise to the same 2D geometry. To demonstrate these ambiguities, we develop new algorithms for fitting a 3D morphable model to 2D landmarks or contours under either orthographic or perspective projection and show how to compute flexibility modes for both cases. We show that both fitting problems can be posed as a separable nonlinear least squares problem and solved efficiently. We demonstrate both quantitatively and qualitatively that the ambiguity is present in reconstructions from geometric information alone but also in reconstructions from a state-of-the-art CNN-based method.

3D Morphable Face Models—Past, Present, and Future

“…This becomes important when a face is close to the camera. At "selfie" distance (e.g., 0.5m), the difference between perspective and orthographic projection of 3D face landmarks is about 6% of the interocular distance [Bas and Smith 2019]. For this reason perspective projection is commonly used in the context of 3DMMs, for example, in the original paper and more recently in a shape-from-landmarks setting [Cao et al 2014a[Cao et al , 2013.…”

“…For this reason perspective projection is commonly used in the context of 3DMMs, for example, in the original paper and more recently in a shape-from-landmarks setting [Cao et al 2014a[Cao et al , 2013. Unfortunately, since calibration information is rarely available the increased complexity of this model introduces ambiguities between shape, scale and focal length that have only recently been studied [Bas and Smith 2019;, though the ambiguity has often been hinted at in the literature. For example, the original paper relied on a fixed, manually provided subject-camera distance.…”

A Morphable Face Albedo Model

“…For this reason, in Section 3.3 we propose a robust algorithm for stitching the photometric views without blurring potentially misaligned features. We provide an implementation of our sampling, weighting and blending pipeline as an extension of the MatlabRenderer toolbox [2].…”

Learning Perspective Undistortion of Portraits

“…Huang et al [31] used a generative model to synthesize an identity-preserving frontal view from a profile. Bas et al [5] proposed an approach for fitting a 3D morphable model to 2D landmarks or contours under either orthographic or perspective projection.…”