Unsupervised Learning of Consensus Maximization for 3D Vision Problems

Abstract

Consensus maximization is a key strategy in 3D vision for robust geometric model estimation from measurements with outliers. Generic methods for consensus maximization, such as Random Sampling and Consensus (RANSAC), have played a tremendous role in the success of 3D vision, in spite of the ubiquity of outliers. However, replicating the same generic behaviour in a deeply learned architecture, using supervised approaches, has proven to be difficult. In that context, unsupervised methods have a huge potential to adapt to any unseen data distribution, and therefore are highly desirable. In this paper, we propose for the first time an unsupervised learning framework for consensus maximization, in the context of solving 3D vision problems. For that purpose, we establish a relationship between inlier measurements, represented by an ideal of inlier set, and the subspace of polynomials representing the space of target transformations. Using this relationship, we derive a constraint that must be satisfied by the sought inlier set. This constraint can be tested without knowing the transformation parameters, therefore allows us to efficiently define the geometric model fitting cost. This model fitting cost is used as a supervisory signal for learning consensus maximization, where the learning process seeks for the largest measurement set that minimizes the proposed model fitting cost. Using our method, we solve a diverse set of 3D vision problems, including 3D-3D matching, non-rigid 3D shape matching with piece-wise rigidity and image-toimage matching. Despite being unsupervised, our method outperforms RANSAC in all three tasks for several datasets.

Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC

“…Alternatively to making RANSAC differentiable, some authors propose to replace RANSAC by a neural network [61], [62], [63], [64], [65]. In these works, the neural network acts as a classifier for model inliers, effectively acting as a robust estimator for model parameters.…”

Wide-Baseline Relative Camera Pose Estimation with Directional Learning

“…Recent works deploy deep learning in subproblems such as feature detection [54,11], filtering or reweighting outliers [64,52]. Differentiable versions of consensus methods like RANSAC have also been proposed [49,3]. These still rely on sufficiently accurate matches, an uncertain prospect in wide-baseline settings.…”

“…Prevailing approaches recover the global model from corresponding points [31,21,43] within an iterative robust model fitting process [16,66]. Recent progress has introduced deeplearned modules that can replace components of this classic pipeline [64,52,11,10,54,45,49,3]. While this class of techniques has been extensively analyzed [51], well-known failure cases include where feature detection or matching is difficult, such as low image overlap, large changes in scale or perspective, or scenes with insufficient or repeated textures.…”

Unsupervised Learning for Robust Fitting:A Reinforcement Learning Approach

“…Our work is closely related to the unsupervised learning approach for consensus maximization proposed by Probst et al [27]. However, we take a different approach by exploiting the tree structure of the underlying fitting problem.…”

“…In addition to popular methods such as Random Sample Consensus (RANSAC) [10] and a number of randomized or deterministic variants [7,6,20,16,2,4,1], the advent of deep learning in recent years has inspired research in learning-based approaches for robust estimation [29,30,22,27,8,18]. The main idea behind these techniques is to exploit the learning capabilities of deep Convolutional Neural Networks (CNNs) to directly regress the robust estimates [18,8], or quickly identify the outliers [22] These approaches have demonstrated their superior performance on many datasets, and hence, developing learningbased robust estimators can be a promising research direction.…”