Surface Networks

Abstract

We study data-driven representations for three-dimensional triangle meshes, which are one of the prevalent objects used to represent 3D geometry. Recent works have developed models that exploit the intrinsic geometry of manifolds and graphs, namely the Graph Neural Networks (GNNs) and its spectral variants, which learn from the local metric tensor via the Laplacian operator.Despite offering excellent sample complexity and built-in invariances, intrinsic geometry alone is invariant to isometric deformations, making it unsuitable for many applications. To overcome this limitation, we propose several upgrades to GNNs to leverage extrinsic differential geometry properties of three-dimensional surfaces, increasing its modeling power. In particular, we propose to exploit the Dirac operator, whose spectrum detects principal curvature directions -this is in stark contrast with the classical Laplace operator, which directly measures mean curvature. We coin the resulting models Surface Networks (SN).We prove that these models define shape representations that are stable to deformation and to discretization, and we demonstrate the efficiency and versatility of SNs on two challenging tasks: temporal prediction of mesh deformations under non-linear dynamics and generative models using a variational autoencoder framework with encoders/decoders given by SNs.A particularly simple and popular extrinsic method [36,37] represents shapes as point clouds in R 3 of variable size, and leverages recent deep learning models that operate on input sets [44,43]. Despite its advantages in terms of ease of data acquisition (they no longer require a mesh triangulation) and good empirical performance on shape classification and segmentation tasks, one may wonder whether this simplification comes at a loss of precision as one considers more challenging prediction tasks.In this paper, we develop an alternative pipeline that applies neural networks directly on triangle meshes, building on geometric deep learning. These models provide data-driven intrinsic graph and manifold representations with inductive biases analogous to CNNs on natural images. Models based on Graph Neural Networks [40] and their spectral variants [6,11,26] have been successfully applied to geometry processing tasks such as shape correspondence [32]. In their basic form, these models learn a deep representation over the discretized surface by combining a latent representation at a given node with a local linear combination of its neighbors' latent representations, and a point-wise nonlinearity. Different models vary in their choice of linear operator and point-wise nonlinearity, which notably includes the graph Laplacian, leading to spectral interpretations of those models.Our contributions are three-fold. First, we extend the model to support extrinsic features. More specifically, we exploit the fact that surfaces in R 3 admit a first-order differential operator, the Dirac operator, that is stable to discretization, provides a direct generalization of Laplacian-based propagati...

ABC: A Big CAD Model Dataset for Geometric Deep Learning

“…Notice that we replace K-ring neighborhoods with K-nearest neighbors for RoSt and Jets to support point cloud input. Also, 7 machine learning methods are selected, including PointNet++ (PN++) [50], Dynamic Graph CNN (DGCNN) [58], Pointwise CNN (PwCNN) [34], PointCNN (PCNN) [41], Laplacian Surface Network (Laplace) [40], PCP-Net (PCPN) [31] and Point Convolutional Neural Networks by Extension Operators (ExtOp) [13]. Of these methods, Laplace operates on triangle mesh input and the rest on point cloud input.…”

“…Graph convolutional neural networks were applied to non-rigid shape analysis in [17,63]. Surface Networks [40] further proposes the use of the differential geometry operators to extend GNNs to exploit properties of the surface. For a number of problems where quantities of interest are localized, spatial filters are more suitable than spectral filters.…”

A-CNN: Annularly Convolutional Neural Networks on Point Clouds

“…Directionally Convolutional Networks (DCN) [41] applies convolution operation on the triangular mesh of 3D shapes to address part segmentation problem by combining local and global features. Lastly, Surface Networks [14] propose upgrades to Graph Neural Networks to leverage extrinsic differential geometry properties of 3D surfaces for increasing their modeling power.…”

Leveraging Shape Completion for 3D Siamese Tracking

“…However, LIDARs generate sparse point clouds, not readily suitable for conventional CNN processing. Most current works pre-process 3D point clouds for use in CNNs by either voxelizing the 3D space [29,21] or by projecting point clouds into a planar space [53,27,10,51]. However, these methods lose fine-grained geometric details.…”