Shape registration in the time of transformers

Abstract

In this paper, we propose a transformer-based procedure for the efficient registration of non-rigid 3D point clouds. The proposed approach is data-driven and adopts for the first time the transformer architecture in the registration task. Our method is general and applies to different settings. Given a fixed template with some desired properties (e.g. skinning weights or other animation cues), we can register raw acquired data to it, thereby transferring all the template properties to the input geometry. Alternatively, given a pair of shapes, our method can register the first onto the second (or vice-versa), obtaining a high-quality dense correspondence between the two. In both contexts, the quality of our results enables us to target real applications such as texture transfer and shape interpolation. Furthermore, we also show that including an estimation of the underlying density of the surface eases the learning process. By exploiting the potential of this architecture, we can train our model requiring only a sparse set of ground truth correspondences (10 ∼ 20% of the total points). The proposed model and the analysis that we perform pave the way for future exploration of transformer-based architectures for registration and matching applications. Qualitative and quantitative evaluations demonstrate that our pipeline outperforms state-of-the-art methods for deformable and unordered 3D data registration on different datasets and scenarios.Preprint. Under review.

Bending Graphs: Hierarchical Shape Matching using Gated Optimal Transport

“…Despite building on top of point cloud features, FMNet [31] finds correspondences densely on deformable shapes. Recent works [21,59] also use learned point cloud features [57] to describe local regions. Other methods also suggest integrating spectral manifold wavelets [25] or iterative spectral upsampling into functional maps [36].…”

“…Several methods have been proposed to solve these issues [18,36,63], but still, the performance deteriorates in a more challenging scenario [35]. Unlike previous works that rely on axiomatic descriptors [39,48], recent methods focus on learning an optimal descriptor to have a better functional map estimation [13,15,24,31,34,49,54,59]. Marin et al [34] propose a two-stage method to estimate an optimal linear transformation by the use of an invariant embedding network combined with a probe function network.…”

“…Marin et al [34] propose a two-stage method to estimate an optimal linear transformation by the use of an invariant embedding network combined with a probe function network. Trappolini et al [59] propose to estimate the transformation between two input point clouds with an auto-encoder architecture and a transformer network [61]. Groueix et al [21] learn to estimate the transformation between two given shapes using a neural network.…”

“…In the first experiment, we use the FAUST dataset [4] with the same testing split used in [34,59]. This dataset contains 100 human shapes with a per-vertex correspondence which allows us to evaluate the dense correspondence quality of our method.…”

“…We compare our method with two dense correspondence methods using an auto-encoder architecture, i.e. 3DC [21] and SRT [59], a functional map method with learned linear high dimensional basis, i.e. LinInv [34], and a method that directly learns optimal descriptors with the functional map framework, i.e.…”