Deep Closest Point: Learning Representations for Point Cloud Registration

Abstract

Point cloud registration is a key problem for computer vision applied to robotics, medical imaging, and other applications. This problem involves finding a rigid transformation from one point cloud into another so that they align. Iterative Closest Point (ICP) and its variants provide simple and easily-implemented iterative methods for this task, but these algorithms can converge to spurious local optima. To address local optima and other difficulties in the ICP pipeline, we propose a learning-based method, titled Deep Closest Point (DCP), inspired by recent techniques in computer vision and natural language processing. Our model consists of three parts: a point cloud embedding network, an attention-based module combined with a pointer generation layer, to approximate combinatorial matching, and a differentiable singular value decomposition (SVD) layer to extract the final rigid transformation. We train our model end-to-end on the ModelNet40 dataset and show in several settings that it performs better than ICP, its variants (e.g., Go-ICP, FGR), and the recently-proposed learning-based method PointNetLK. Beyond providing a state-of-the-art registration technique, we evaluate the suitability of our learned features transferred to unseen objects. We also provide preliminary analysis of our learned model to help understand whether domain-specific and/or global features facilitate rigid registration.

Optimal, Non-Rigid Alignment for Feature-Preserving Mesh Denoising

“…The energies we minimize are very local and cannot establish correspondences between points far apart. Fortunately, it appears that modern methods for rigid alignment [24,32] make this assumption reasonably easy to justify. Our second assumption, in case of the GLP based energy, is that we have triangle mesh connectivities for both the combined point cloud and for each sub-scan.…”

Under the Radar: Learning to Predict Robust Keypoints for Odometry Estimation and Metric Localisation in Radar

“…By embedding a differentiable point-based pose estimator [9] learns to predict keypoint locations for the task of rotation prediction; however the formulation predicts only object category specific keypoints which cannot generalise to new scenes. Conversely [10] registers two point clouds by predicting point-wise descriptors for matching, followed by the same pose estimation formulation.…”

“…Inspired by [9], [10] we learn to predict keypoints specialised for localisation by embedding a pose estimator in our architecture and use only pose information as supervision. This avoids imposing any assumptions on what makes suitable keypoints and enables pose prediction at any angle, a limitation of the current state-of-the-art [13].…”

“…Similar to [9], [10] given matched keypoints and weights we calculate the transform between them using singular value decomposition (SVD) as laid out in Algorithm 2 (detailed further in [16]). Crucially this pose estimation is differentiable, allowing us to backpropagate from transform error right through to the keypoint prediction network.…”

When Deep Learning Meets Data Alignment: A Review on Deep Registration Networks (DRNs)

“…The pipeline is decomposed into three main components: 'Learning Shape Descriptor' with a MultiLayer Perceptron (MLP) that learns descriptors from the input source and target point sets; 'Coherent PointMorph' that is a three MLPs block fed with the two descriptors concatenated with the source data points; and the 'Point Set Alignment', where the loss function is defined to determine the quality of the alignment. Deep Closest Point [70] registers two point clouds by first embedding them into high-dimensional space using DGCNN [101] to extract features. After that, contextual information is estimated using an attention-based module that provides a dependency term between the feature sets, i.e., one set is modified in a way that is knowledgeable about the structure of the other.…”