3D Point Cloud Registration with Multi-Scale Architecture and Unsupervised Transfer Learning

Abstract

We present MS-SVConv, a fast multi-scale deep neural network that outputs features from point clouds for 3D registration between two scenes. We compute features using a 3D sparse voxel convolutional network on a point cloud at different scales and then fuse the features through fullyconnected layers. With supervised learning, we show significant improvements compared to state-of-the-art methods on the competitive and well-known 3DMatch benchmark. We also achieve a better generalization through different source and target datasets, with very fast computation. Finally, we present a strategy to fine-tune MS-SVConv on unknown datasets in a self-supervised way, which leads to state-of-the-art results on ETH and TUM datasets. The code is publicly available at: https://github.com/ humanpose1/MS-SVConv.

IMFNet: Interpretable Multimodal Fusion for Point Cloud Registration

“…All (s) Time (s) FCGF [5] 25.06 0.06 PREDATOR [12] 762.58 0.47 SpinNet [1] 17155.5 39.62 MS-SVConv [11] 41.23 0.10 Ours 39.98 0.09 Table 3. Running speed comparison on 3DMatch.…”

“…SpinNet [1] and MS-SVConv [11]. As shown in Table 4, the proposed IMFNet also achieves the state-of-the-art feature match recall at the registration dataset with low-overlap point clouds.…”

“…SpinNet [1] projected the point clouds into a spherical space and used spherical convolution operations to extract the structural information as the point descriptor. MS-SVConv [11] designed a multi-scale framework to learn the descriptor by exploring the multi-scale structure information in describing the points.…”

“…Computation efficiency comparison. We compared the single descriptors' extraction speed with the recent FCGF [5], PREDATOR [12], SpinNet [1] and MS-SVConv [11] on the 3DMatch testing dataset. Table 3 shows that our method obtains comparable efficiency to FCGF and is faster than the recent PREDATOR, SpinNet and MS-SVConv.…”

“…This technique played a critical role in many applications, including robotics and augmented reality [14]. Among the existing registration methods [1,5,[11][12][13], descriptor-based methods [1,11] are an important category and achieve the state-of-the-art accuracy in the large-scale real-world datasets (e.g., 3DMatch [26]). The distinctiveness of the 3D point descriptor dominates the performance of these descriptor-based registration methods.…”