Geometric Feature Learning for 3D Meshes

Abstract

Geometric feature learning for 3D meshes is central to computer graphics and highly important for numerous vision applications. However, deep learning currently lags in hierarchical modeling of heterogeneous 3D meshes due to the lack of required operations and/or their efficient implementations. In this paper, we propose a series of modular operations for effective geometric deep learning over heterogeneous 3D meshes. These operations include mesh convolutions, (un)pooling and efficient mesh decimation. We provide open source implementation of these operations, collectively termed Picasso. The mesh decimation module of Picasso is GPU-accelerated, which can process a batch of meshes on-the-fly for deep learning. Our (un)pooling operations compute features for newly-created neurons across network layers of varying resolution. Our mesh convolutions include facet2vertex, vertex2facet, and facet2facet convolutions that exploit vMF mixture and Barycentric interpolation to incorporate fuzzy modelling. Leveraging the modular operations of Picasso, we contribute a novel hierarchical neural network, PicassoNet-II, to learn highly discriminative features from 3D meshes. PicassoNet-II accepts primitive geometrics and fine textures of mesh facets as input features, while processing full scene meshes. Our network achieves highly competitive performance for shape analysis and scene parsing on a variety of benchmarks. We release Picasso and PicassoNet-II on Github.

Learning Self-Prior for Mesh Inpainting Using Self-Supervised Graph Convolutional Networks

“…Additionally, we are interested in leveraging vertex colors alongside vertex positions, as most off-theshelf 3D scanners capture both. Lastly, although potential ideas have been proposed by recent studies [69], [70], [71], improving the time and memory efficiency of basic neural network layers (e.g., convolution and dynamic pooling/unpooling) for meshes remains an important and challenging research direction.…”

Learning Self-prior for Mesh Denoising Using Dual Graph Convolutional Networks

“…Additionally, we are interested in leveraging vertex colors alongside vertex positions, as most off-the-shelf 3D scanners capture both. Lastly, although recent studies [66], [67], [68] have proposed potential ideas, improving the time and memory efficiency of basic neural network layers (e.g., convolution and dynamic pooling/unpooling) for meshes remains an important and challenging research direction.…”

Improving Three-Dimensional Building Segmentation on Three-Dimensional City Models through Simulated Data and Contextual Analysis for Building Extraction

“…Artificial intelligence (AI) provides many possibilities with deep learning using neural networks. According to the literature, the use of convolutional neural networks (CNNs) delivers the best segmentation results [7,[11][12][13][14][15][16].…”

“…Three-dimensional models have the advantage of possessing geometric information and spectral information from the red, green, and blue (RGB) bands. The integration of spectral information can improve segmentation by, for example, using surface color to differentiate objects [11][12][13][14][15][16][17]. This information is very important for separating objects that have obvious color discontinuities but great geometric continuity, such as houses arranged in rows that have different siding color.…”

“…The training data encompass both actual data representing Quebec City, Canada, and synthetic data generated from diverse platforms like the 3D engine "Unreal Engine" and Evermotion. PicassoNet-II, a semantic segmentation architecture tailored for textured meshes, was developed nearly two years ago [15]. Initially, the semantic segmentation of 3D cities was executed using this model.…”