Globally-Optimal Inlier Set Maximisation for Simultaneous Camera Pose and Feature Correspondence

Abstract

Estimating the 6-DoF pose of a camera from a single image relative to a pre-computed 3D point-set is an important task for many computer vision applications. Perspective-n-Point (PnP) solvers are routinely used for camera pose estimation, provided that a good quality set of 2D-3D feature correspondences are known beforehand. However, finding optimal correspondences between 2D key-points and a 3D point-set is non-trivial, especially when only geometric (position) information is known. Existing approaches to the simultaneous pose and correspondence problem use local optimisation, and are therefore unlikely to find the optimal solution without a good pose initialisation, or introduce restrictive assumptions. Since a large proportion of outliers are common for this problem, we instead propose a globally-optimal inlier set cardinality maximisation approach which jointly estimates optimal camera pose and optimal correspondences. Our approach employs branchand-bound to search the 6D space of camera poses, guaranteeing global optimality without requiring a pose prior. The geometry of SE(3) is used to find novel upper and lower bounds for the number of inliers and local optimisation is integrated to accelerate convergence. The evaluation empirically supports the optimality proof and shows that the method performs much more robustly than existing approaches, including on a large-scale outdoor data-set.

Outlier-Robust Spatial Perception: Hardness, General-Purpose Algorithms, and Guarantees

“…But when the outliers' number is more than 50%, RANSAC tends to be slow, and brittle [10], [18]. Thereby, recent approaches adopt either robust cost functions [46], [9], [25], or BnB [24], [47], [18]: Zhou et al [25] propose fast global registration (FGR), which is based on the Geman-McClure robust cost function; Yang et al [24] propose a approach; Campbell et al [47] employ BnB to search the space of camera poses, guaranteeing global optimality without requiring a pose prior; and Bustos et al [18] add a pre-processing step, that removes gross outliers before RANSAC or BnB. Other approaches, that iteratively compute point correspondences, include iterative closest point (ICP) [48], [49], and trimmed iterative closest point algorithm [50]; all require an accurate initial guess [25].…”

A Quaternion-Based Certifiably Optimal Solution to the Wahba Problem With Outliers

“…Global Methods. The most popular class of global methods for robust rotation search is based on Consensus Maximization [19] and branch-and-bound (BnB) [16]. Hartley and Kahl [29] first proposed using BnB for rotation search, and Bazin et al [7] adopted consensus maximization to extend their BnB algorithm with a robust formulation.…”

The Alignment of the Spheres: Globally-Optimal Spherical Mixture Alignment for Camera Pose Estimation

“…However this requires knowing the true outlier fraction in advance; if incorrectly specified, the optimum may not occur at the correct pose. Campbell et al [11,12] proposed a globally-optimal inlier set cardinality maximization solution to the problem. While robust, this objective function is discrete and challenging to optimize, and operates on sampled points instead of the underlying surfaces.…”