VV-Net: Voxel VAE Net With Group Convolutions for Point Cloud Segmentation

Abstract

We present a novel algorithm for point cloud segmentation. Our approach transforms unstructured point clouds into regular voxel grids, and further uses a kernel-based interpolated variational autoencoder (VAE) architecture to encode the local geometry within each voxel. Traditionally, the voxel representation only comprises Boolean occupancy information which fails to capture the sparsely distributed points within voxels in a compact manner. In order to handle sparse distributions of points, we further employ radial basis functions (RBF) to compute a local, continuous representation within each voxel. Our approach results in a good volumetric representation that effectively tackles noisy point cloud datasets and is more robust for learning. Moreover, we further introduce group equivariant CNN to 3D, by defining the convolution operator on a symmetry group acting on Z 3 and its isomorphic sets. This improves the expressive capacity without increasing parameters, leading to more robust segmentation results. We highlight the performance on standard benchmarks and show that our approach outperforms state-of-the-art segmentation algorithms on the ShapeNet and S3DIS datasets.

PRS-Net: Planar Reflective Symmetry Detection Net for 3D Models

“…Wang and Solomon [46] propose a learningbased method named Deep Closest Point (DCP) to predict rigid transformation for point cloud registration, and a Partial Registration Network (PRNet) [47] for partial-topartial registration. Meng et al [48] propose a network which applies group convolutions on regular voxel grids and encodes features computed from radial basis functions (RBF), for point cloud segmentation, achieving state-ofthe-art results. Besides, there are some methods that deal with graphs and mesh representations.…”

Deep FusionNet for Point Cloud Semantic Segmentation

“…Other studies [11,23,29,36,54] voxelize point clouds into 3D volumetric voxels and apply 3D convolution networks for point cloud processing and understanding. Among these studies, VV-Net [30] uses a kernel-based interpolated variational auto encoder (VAE) on regular voxel grids; instance segmentation models have been proposed on dense 3D voxels/2D grids [19,61]; panoptic labels can be predicted using a spatially hashed volumetric map [31]; high-resolution RGB inputs have been leveraged by associating 2D images with the volumetric grid [11]; efficient feature aggregations have been explored in PVCNN [27] through a voxelbased regular CNN in low resolution to avoid random memory accesses.…”

A survey on deep geometry learning: From a representation perspective

“…Mean intersection-overunion (mIOU) is often used as the evaluation metric for shape segmentation. Most researchers choose to use point-based representation for the segmentation task [6,7,24,58,61].…”