Meshlet Priors for 3D Mesh Reconstruction

Abstract

Estimating a mesh from an unordered set of sparse, noisy 3D points is a challenging problem that requires carefully selected priors. Existing hand-crafted priors, such as smoothness regularizers, impose an undesirable trade-off between attenuating noise and preserving local detail. Recent deep-learning approaches produce impressive results by learning priors directly from the data. However, the priors are learned at the object level, which makes these algorithms class-specific, and even sensitive to the pose of the object. We introduce meshlets, small patches of mesh that we use to learn local shape priors. Meshlets act as a dictionary of local features and thus allow to use learned priors to reconstruct object meshes in any pose and from unseen classes, even when the noise is large and the samples sparse.

Meshing Point Clouds with Predicted Intrinsic-Extrinsic Ratio Guidance

“…Since the estimation of local connectivity relies more on local inductive biases, which encourages the generalizability. Similar phenomenons are also observed in [27,1,31]. Although our method proposes O(k 2 n) candidate triangles, the candidate classification can be processed in batch.…”

“…However, unlike 2D images, 3D polygon meshes are irregular geometric formats and are difficult to be directly generated from the neural networks. Existing learningbased mesh generative methods mainly follow five paradigms: deformation-based methods [43,18,14,35,29,23,16,44], folding-based methods [46,19,10], primitivebased methods [6,41,39], optimization-based methods [1,45], and implicit-fieldfunction-based methods [36,7,15,32,26].…”

“…As for the implicit-field-functionbased methods, they employ neural networks to learn an implicit field function and then utilize marching cube algorithms [30] to extract the iso-surface, and thus face similar problems as the traditional implicit reconstruction methods. There are also some recent optimization-based methods, which either utilize a deep neural network as local geometric prior [45] or learn some local shape priors from the data [1]. They formulate mesh reconstruction as an optimization problem and are thus computational expensive.…”

“…Unlike 2D images and 3D voxels, polygon meshes is an irregular geometric representation, which prevents it from being generated by the neural network directly. However, there are still lots of attempts to explore the neural-networkcompatible representations for mesh generation, including template meshes with deformation [43,18,14,35,29,23,16,44], 2D squares with folding [46,19,10], primitives with assembly [6,41,39], implicit field function [36,7,15,32], and meshlets with optimization [1,45]. Existing learning-based methods typically follow the "encoder-decoder" paradigm.…”