Efficient Learning on Point Clouds With Basis Point Sets

Abstract

With an increased availability of 3D scanning technology, point clouds are moving into the focus of computer vision as a rich representation of everyday scenes. However, they are hard to handle for machine learning algorithms due to their unordered structure. One common approach is to apply occupancy grid mapping, which dramatically increases the amount of data stored and at the same time loses details through discretization. Recently, deep learning models were proposed to handle point clouds directly and achieve input permutation invariance. However, these architectures often use an increased number of parameters and are computationally inefficient. In this work we propose basis point sets (BPS) as a highly efficient and fully general way to process point clouds with machine learning algorithms. The basis point set representation is a residual representation that can be computed efficiently and can be used with standard neural network architectures and other machine learning algorithms. Using the proposed representation as the input to a simple fully connected network allows us to match the performance of PointNet on a shape classification task, while using three orders of magnitude less floating point operations. In a second experiment, we show how the proposed representation can be used for registering high resolution meshes to noisy 3D scans. Here, we present the first method for single-pass high-resolution mesh registration, avoiding time-consuming per-scan optimization and allowing real-time execution.

A Review of Body Measurement Using 3D Scanning

“…An example of such an approach is done by Kwok et al [88], consisting of iteratively selecting the mesh from the statistical poseshape space and fitting the clothes to match the input 3D scan. Prokudin et al [131] propose a deep learning model for template fitting, supervised by SMPL templates fitted to the dataset before learning. The learning is based on the distances between the set of 3D scan features, called the basis point set, and the ground truth template mesh.…”

Reconstructing Human Body Mesh from Point Clouds by Adversarial GP Network

“…Intra (cm) Inter (cm) 3D-CODED [20] 1.985 2.878 Stitched puppets [8] 1.568 3.126 LBS-AE [16] 2.161 4.08 FARM [18] 2.810 4.123 BPS [19] 2.327 4.529 FMNet [13] 2.436 4.826 Convex-Opt [41] 4.860 8.304 Our GP 2.349 2.734 Our Adversarial GP 1.904 2.759 Table 2. Results for the FAUST intra-and inter-subject challenges for human body registration.…”

“…Deep Hierarchical Networks [17] learn a 3D human body embedding which can then be fitted to data, leveraging a set of manually selected landmarks. Basis Point Sets [19] propose an efficient point cloud encoding, which can then be combined with DNNs [28] for shape registration and completion tasks.…”

“…Many of these learning approaches rely on some form of human a priori knowledge. Most methods propose matching to an explicit shape deformation model, for which a reduced parameterization is predicted [15][16][17] or whose mapping to the data is learned [18,19] from observations.…”

Grasping Field: Learning Implicit Representations for Human Grasps

“…Then, the predicted hands are optimized together with the object meshes to refine the contact points. While in [84], the parameters are generated directly from the given Basis Point Set [66] of the objects. Our work differs from the previous works in that, by also considering the object distance field, we propose a learnable representation for modelling hand-object interaction that can be used without contact post-processing.…”