Deformation transfer of 3D human shapes and poses on manifolds

Abstract

In this paper, we introduce a novel method to transfer the deformation of a human body to another directly on a manifold. There exists a rich literature on transferring deformations based on Euclidean representations. However, a 3D human shape and pose live on a manifold and have a Riemannian structure. The proposed method uses the Lie Bodies manifold representation of 3D triangulated bodies. Its benefits are preserved, namely, minimum required degrees of freedom for any triangle deformation and no heuristics to constrain excessive ones. We give a closed form solution for deformation transfer directly on the Lie Bodies. The deformations have strictly positive determinants ensuring that nonphysical deformations are removed. We show examples on three datasets, and highlight differences with the Euclidean deformation transfer.

Deformation transfer survey

“…Virtualization of negative space can be achieved by adding virtual triangles and deformation smoothness can be encoded either implicitly through an alternative deformation representation (like the dual-mesh in [25] ) or explicitly through a regularization term (like the Laplacian in [16] ). Finally, most recently, Shabayek et al [10] have introduced a non-Euclidean deformation representation that avoids degenerate transformations by design.…”

“…In another work Shabayek et al [10] observe that Euclidean spaces can be ineffective for representing deformation. They adapt the Lie Bodies representation [11] to decompose triangle deformations into a group of transformations expressed using a Lie Algebra.…”

“…As well as enabling broader application, their formulation has another advantage in that each deformation is expressed in a local coordinate system that more easily enables post-processing (they apply a Poisson interpolation post-process to improve temporal properties of transferred sequences). As another example, Shabayek et al [10] employ a deformation representation that encodes each triangle deformation as a group containing a rotation, in-place deformation, and anisotropic scaling. By projecting these transformation groups into a special type of manifold that features a Riemannian metric, they can produce results that appear more natural than those of seminal deformation transfer.…”

3d Deformation Signature for Dynamic Face Recognition

“…A 3D face represented by a triangulated mesh contains N triangles. In this work, 3D triangulated meshes are described using the Lie Bodies manifold representation [13,14] which defines a deformation that does not suffer from the possibility of having negative determinants, unlike Euclidean deformations, which represent non-physical deformations. Without loss of generality, any non-degenerate triangle can be represented by its edge matrix…”

“…Hence, the way any 3D face deforms is unique given its shape and geometry. The proposed 3DS is defined in terms of a robust non-linear 6D-space representation [13,14] which guarantees, by construction, physically plausible 3D deformations. The contribution of this work is a novel 3DS for a temporal face sequence instance.…”