Convolution on Rotation-Invariant and Multi-Scale Feature Graph for 3D Point Set Segmentation

Abstract

Invariance against rotation of 3D objects is one of the essential properties for 3D shape analysis. Recently proposed algorithms (e.g., [9], [10], [12], [13]) have achieved rotationally invariant 3D point set analysis by using inherently rotation-invariant 3D shape features, i.e., distances and angles among 3D points, as input to Deep Neural Networks (DNNs). The DNNs capture spatial hierarchy and context among the geometric features to produce accurate analytical results. In this paper, we delve further into the DNN-based approach to rotation-invariant and highly accurate 3D point set analysis. In particular, we focus our attention on segmentation of 3D point sets, which is one of the most challenging among 3D point set analysis tasks. We propose a novel DNN for 3D point set segmentation called Rotation-invariant and Multi-scale feature Graph convolutional neural network, or RMGnet. Our RMGnet is more flexible than the previous methods as it accepts as input any handcrafted 3D shape features having rotation invariance. In addition, to accurately segment 3D point sets composed of parts having various sizes, we randomize scales at which handcrafted features are extracted and perform multi-resolution analysis of the features by using the DNN. Experimental evaluation demonstrates high segmentation accuracy as well as rotation invariance of the proposed RMGnet.

MaskLRF: Self-Supervised Pretraining via Masked Autoencoding of Local Reference Frames for Rotation-Invariant 3D Point Set Analysis

“…However, these methods are not rigorously rotation invariant since they employ non-RI DNNs. Recently, Spezialetti et al [50] and Furuya et al [51] introduced selfsupervised learning of fully rotation-invariant 3D point set features. However, their methods are designed not for pretraining, but for a specific task such as registration or retrieval.…”

“…They are six MPM-based methods [6], [7], [8], [9], [10], [11]. In addition, previously proposed DNNs having rotation invariance [35], [36] 1 , [37], [39], [40], [42], [43], [45], [47], [60] are included in the set of competitors. These RI DNNs are not pretrained, but are trained from scratch for each downstream task.…”

“…Furthermore, I add pretrained RI DNNs to the competitors for a fair evaluation. Specifically, the two state-of-the-art RI DNNs [36], [47] are pretrained by using the state-of-the-art selfsupervised representation learning algorithm for 3D point set called SDMM [51]. SDMM is built upon the self-distillation framework [61] originally proposed for 2D images.…”

Multi-Head Attentional Point Cloud Classification and Segmentation Using Strictly Rotation-Invariant Representations

“…The specific rotation transformation they used could cause the same rotation transformation to all intermediate-layer quaternion features. Furuya et al [24] published a rotation-invariant multi-scale framework for any manual 3D graph features with rotation invariance, based on the relations between 3D points and their normal vectors. Chen et al [25] designed ClusterNet [37] for strictly pointwise rotation-invariant representation by rigorously rotation invariant (RRI).…”

“…However, they were exposed to lower accuracy, which prompted the research development. RMGNet [24] constructed a multi-scale graph convolutional neural network for segmentation through the handcrafted rotation-invariant features. In addition, ClusterNet [25] and SRINet [26] were designed for strictly rotation-invariant representations through point projection.…”