Variational Autoencoders for Localized Mesh Deformation Component Analysis

Abstract

Spatially localized deformation components are very useful for shape analysis and synthesis in 3D geometry processing. Several methods have recently been developed, with an aim to extract intuitive and interpretable deformation components. However, these techniques suffer from fundamental limitations especially for meshes with noise or large-scale nonlinear deformations, and may not always be able to identify important deformation components. In this paper we propose a mesh-based variational autoencoder architecture that is able to cope with meshes with irregular connectivity and nonlinear deformations, assuming that the analyzed dataset contains meshes with the same vertex connectivity, which is common for deformation analysis. To help localize deformations, we introduce sparse regularization in this framework, along with spectral graph convolutional operations. Through modifying the regularization formulation and allowing dynamic change of sparsity ranges, we improve the visual quality and reconstruction ability of the extracted deformation components. Our system also provides a nonlinear approach to reconstruction of meshes using the extracted basis, which is more effective than the current linear combination approach. As an important application of localized deformation components and a novel approach on its own, we further develop a neural shape editing method, achieving shape editing and deformation component extraction in a unified framework, and ensuring plausibility of the edited shapes. Extensive experiments show that our method outperforms state-of-the-art methods in both qualitative and quantitative evaluations. We also demonstrate the effectiveness of our method for neural shape editing.

DeepMetaHandles: Learning Deformation Meta-Handles of 3D Meshes with Biharmonic Coordinates

“…However, these approaches are not able to produce fine-grained geometric details due to the limit of the resolution.While mesh is a preferable representation, generating meshes is very challenging in particular when preventing the generation of nonmanifold faces or disconnected components [29]. Hence, previous work, such as the one of Tan et al [36,35], considers generating novel shapes by deforming a given template mesh, limiting the scope of the generation to the possible variations of the template shape. We propose to overcome this limitation with our conditional generative model, which takes any 3D mesh as input to deform.…”

“…For such reasons, to create new meshes, instead of generating a mesh from scratch, recent work assumes that the connectivity structure of geometries is known so that the creation space is restricted to changing the geometry without altering the structure. For example, [36,35] create new shapes by deformations of one template mesh. They, however, limit the scope of the shape generation to possible variants of the template mesh.…”

A Revisit of Shape Editing Techniques: from the Geometric to the Neural Viewpoint

“…To solve this, graph-based convolutions [236] and mesh pooling [237] have been introduced. At the same time, they [238] propose a convolutional mesh autoencoder utilizing the locality of the convolution operator and sparsity constraints to extract the local deformation components of the deformable shapes. The deformation components can be used to synthesize new shapes.…”

A Survey on Deep Geometry Learning: From a Representation Perspective

“…In order to overcome the problem that deformation gradient cannot work well in large-scale deformation, Gao et al [20] designed an as-consistent-as-possible (ACAP) representation to constrain the rotation angle and rotation axes between adjacent vertices in the deformable mesh. Tan et al [78] proposed the SparseAE based on the ACAP representation [20], which applies graph convolutional operators [15] to the network.…”