SampleNet: Differentiable Point Cloud Sampling

Abstract

There is a growing number of tasks that work directly on point clouds. As the size of the point cloud grows, so do the computational demands of these tasks. A possible solution is to sample the point cloud first. Classic sampling approaches, such as farthest point sampling (FPS), do not consider the downstream task. A recent work showed that learning a task-specific sampling can improve results significantly. However, the proposed technique did not deal with the non-differentiability of the sampling operation and offered a workaround instead.We introduce a novel differentiable relaxation for point cloud sampling. Our approach employs a soft projection operation that approximates sampled points as a mixture of points in the primary input cloud. The approximation is controlled by a temperature parameter and converges to regular sampling when the temperature goes to zero. During training, we use a projection loss that encourages the temperature to drop, thereby driving every sample point to be close to one of the input points.This approximation scheme leads to consistently good results on various applications such as classification, retrieval, and geometric reconstruction. We also show that the proposed sampling network can be used as a front to a point cloud registration network. This is a challenging task since sampling must be consistent across two different point clouds. In all cases, our method works better than existing non-learned and learned sampling alternatives. Our code is publicly available 1 .

A survey on end‐to‐end point cloud learning

“…And the characteristics of point cloud data are considered. SampleNet [26] perform a differentiable acquisition to approximate point cloud sampling and incorporating a soft projection operation changes the representation by using the local position weight value coordinates in the initial point cloud. Refs.…”

“…Refs. [23] and [26] perform novel feature extraction methods to achieve more effect results. Point-BETR [27] combine point cloud and Transformer structure from natural language processing to encode points, build masked language modelling for self-supervised training, and translate point clouds into language-like words.…”

“…pointNet [11], PointNet++ [22], PointAugment [25], PSTNet [30] use 3D structural features of point cloud data to mine and propose new models, such as the distinction between spatial and temporal dimensions of point cloud data, and the combination of 3D data spatial dimension and 2D data augmentation. AdvectiveNet [23], SampleNet [26], P4Transformer [29], PSTNet [30], HandVoxNet [33], PVN3D [35] extract features from the feature extraction method to achieve more accurate local features in point cloud. Joint depth-pose pointcontrast [24] creates a new loss function to reduce point overlap in point cloud.…”

“…PointAugment [25] performed data fusion of the enhanced 2D data with the 3D data. SampleNet [26] added a projection operation to transform the data and achieved a new point cloud sampling. LU-Net [50] extracted the point cloud data with highlevel features and embedded 3D local features into 2D to obtain the new point cloud data.…”

Deep Learning for 3D Reconstruction, Augmentation, and Registration: A Review Paper

“…SampleNet [ 171 ]: SampleNet is a differentiable sampling network used for reconstruction and classification tasks in point clouds [ 198 ]. It introduces a differentiable relaxation for point cloud sampling by approximating sampled points as a mixture of points in the original point cloud.…”