Interpolated Convolutional Networks for 3D Point Cloud Understanding

Abstract

Point cloud is an important type of 3D representation. However, directly applying convolutions on point clouds is challenging due to the sparse, irregular and unordered data structure. In this paper, we propose a novel Interpolated Convolution operation, InterpConv, to tackle the point cloud feature learning and understanding problem. The key idea is to utilize a set of discrete kernel weights and interpolate point features to neighboring kernel-weight coordinates by an interpolation function for convolution. A normalization term is introduced to handle neighborhoods of different sparsity levels. Our InterpConv is shown to be permutation and sparsity invariant, and can directly handle irregular inputs. We further design Interpolated Convolutional Neural Networks (InterpCNNs) based on Inter-pConv layers to handle point cloud recognition tasks including shape classification, object part segmentation and indoor scene semantic parsing. Experiments show that the networks can capture both fine-grained local structures and global shape context information effectively. The proposed approach achieves state-of-the-art performance on public benchmarks including ModelNet40, ShapeNet Parts and S3DIS.

Deep FusionNet for Point Cloud Semantic Segmentation

“…Similarly, Mao et al . propose to interpolate discrete convolutional weights for convolutions on points [28].…”

“…The knowledge can be easily transferred to the design and training of deep network models for 3D point clouds. Regular voxel-based convolutions are also more efficient than many existing point-wise convolution models [10,25,28,32,50,53,55]. These models requires extra memory and computations to learn or interpolate the weight kernels of the point convolutions, as well as to locate the neighborhood points from irregular point cloud by KNN search or ball querying.…”

“…There is also plenty of work on point-based convolutions that explores the idea of convolutions directly on irregular points [8,25,26,46,47,55]. They learn to approximate a weight function or interpolate convolutional weights [10,25,28,32,50,53,55]. Compared to volumetric convolutions that can directly index a kernel with fixed relative positions of the neighborhoods, the positions of neighbors become unpredictable in these point-wise convolutions as the points are scattered irregularly.…”

3D Capsule Hand Pose Estimation Network Based on Structural Relationship Information

“…In recent years, several studies have explored local structures to enhance feature learning [27][28][29] or project irregular points into a regular space to apply traditional CNN [30,31]. Considering the importance of the local characteristics of point clouds, other recent methods such as self-organizing networks (SO-Net) [29], similarity group proposal networks (SGPN) [32] and PointCNN [33] combine the spatial distribution of the inputted point clouds.…”

KVGCN: A KNN Searching and VLAD Combined Graph Convolutional Network for Point Cloud Segmentation

“…However, projection causes information loss, and the 3D convolution after voxelization brings greater computational cost. Another type of work, on the other hand, uses the original point clouds as input and conducts end-to-end learning, such as the pioneering work PointNet [11], the upgraded PointNet++ [12], and other new deep networks [13][14][15][16][17][18][19][20]. They all directly deal with the unordered point clouds, and learn global and local features to realize the classification and segmentation.…”