Probabilistic Permutation Synchronization Using the Riemannian Structure of the Birkhoff Polytope

Abstract

We present an entirely new geometric and probabilistic approach to synchronization of correspondences across multiple sets of objects or images. In particular, we present two algorithms: (1) Birkhoff-Riemannian L-BFGS for optimizing the relaxed version of the combinatorially intractable cycle consistency loss in a principled manner, (2) Birkhoff-Riemannian Langevin Monte Carlo for generating samples on the Birkhoff Polytope and estimating the confidence of the found solutions. To this end, we first introduce the very recently developed Riemannian geometry of the Birkhoff Polytope. Next, we introduce a new probabilistic synchronization model in the form of a Markov Random Field (MRF). Finally, based on the first order retraction operators, we formulate our problem as simulating a stochastic differential equation and devise new integrators. We show on both synthetic and real datasets that we achieve high quality multi-graph matching results with faster convergence and reliable confidence/uncertainty estimates.

Learning Multiview 3D Point Cloud Registration

“…Taking a step further, other modern methods make use of the global cycle-consistency and optimize only over the poses starting from an initial set of pairwise maps. This efficient approach is known as synchronization [10,61,2,56,3,5,43,69,7,35]. Global structurefrom-motion [17,70] aims to synchronize the observed relative motions by decomposing rotation, translation and scale components.…”

Multi-frame Motion Segmentation by Combining Two-Frame Results

“…States are usually represented by elements of a group, such as the set of permutations or the set of Euclidean transformations. The former can represent local labels of a set of features, as it occurs in multi-view matching applications (Pachauri et al 2013;Birdal and Simsekli 2019). The latter can represent camera reference frames (e.g., in the context of structure from motion (Govindu 2004;Hartley et al 2011Hartley et al , 2013 or pose graph optimization (Carlone et al 2015;Rosen et al 2017)), or local coordinates of 3D point clouds when dealing with 3D registration (Torsello et al 2011;Bernard et al 2015;Arrigoni et al 2016).…”

Riemannian Functional Map Synchronization for Probabilistic Partial Correspondence in Shape Networks

“…( 7), or its Riemannian version Eq. ( 9) as done repeatedly in the literature, we propose to characterize the empirical posterior distribution related to the energy similar to [4]. We formulate the functional map synchronization problem in a probabilistic fashion: We treat the pairwise functional maps as observed random variables and the absolute ones as latent random variables.…”

“…the negative log-posterior is non-convex) and any algorithm that aims to solve one of these problems should be able to operate in the particular manifold of this problem. To this end, we adopt an algorithmically simple yet effective algorithm, the Riemannian Langevin Monte Carlo as explained in [4,13].…”

“…When P is total, e.g. for M = N , it can be relaxed to be a doubly stochastic matrix [4,11], and our case of partial matching can be restricted to the set of stochastic (multinomial) matrices S:…”