Neural Mesh Flow: 3D Manifold Mesh Generation via Diffeomorphic Flows

Abstract

Meshes are important representations of physical 3D entities in the virtual world. Applications like rendering, simulations and 3D printing require meshes to be manifold so that they can interact with the world like the real objects they represent. Prior methods generate meshes with great geometric accuracy but poor manifoldness. In this work we propose Neural Mesh Flow (NMF) to generate two-manifold meshes for genus-0 shapes. Specifically, NMF is a shape auto-encoder consisting of several Neural Ordinary Differential Equation (NODE) [1] blocks that learn accurate mesh geometry by progressively deforming a spherical mesh. Training NMF is simpler compared to state-of-the-art methods since it does not require any explicit mesh-based regularization. Our experiments demonstrate that NMF faciliates several applications such as single-view mesh reconstruction, global shape parameterization, texture mapping, shape deformation and correspondence. Importantly, we demonstrate that manifold meshes generated using NMF are better-suited for physically-based rendering and simulation. Code and data will be released 1 .Preprint. Under review.

A Deep-Learning Approach For Direct Whole-Heart Mesh Reconstruction

“…Therefore, by sampling a large number of points in the physical space, these methods can also achieve high-resolution reconstruction that are not constrained by the voxel resolution of the input image or GPU memory. However, the inference process for such methods is computationally expensive (Gupta and Chandraker, 2020) as it requires prediction on a large number of points. In contrast, our method represents the mesh as a graph (i.e., a sparse matrix) and takes less than a second to predict a high resolution whole heart mesh.…”

MongeNet: Efficient Sampler for Geometric Deep Learning

“…Training a deep learning model requires extensive use of sampling, and faster methods are needed. Indeed for a large number of applications [45,17,14], this sampling is performed online and at each training step when a new mesh is predicted by the neural network.…”

“…Recently, computer vision researchers have demonstrated an increasing interest in developing deep learning models for 3D data understanding [42,5]. As successful applications of those models, we can mention single view object shape reconstruction [45,14], shape and pose estimation [22], point cloud completion and approximation [16], and brain cortical surface reconstruction [38].…”

“…In essence, the mainstream approach relies on sampling point clouds from any given meshes to be able to compute point-based distances such as the Chamfer distance [3] or earth mover's distance [37]. While there exist in-depth discussions about these distance metrics in the literature [13,27], the mesh sampling technique itself remains uncharted, where the uniform sampling approach is widely adopted by most of the 3D deep learning models [45,31,14]. This approach is computationally efficient and is the only mesh sampling method available in existing 3D deep learning libraries like PyTorch3D [35] and Kaolin [19].…”

3DSNet: Unsupervised Shape-to-Shape 3D Style Transfer

“…As point clouds and meshes become the most popular representations for several vision and graphics applications, different approaches have been proposed to compress and reconstruct 3D models. Current shape processing approaches [10,52,51,12] often adopt an autoencoder architecture, where the encoder projects the input family into a lower-dimensional latent space and the decoder reconstructs the data from the latent embedding.…”