A new minimal solution to the relative pose of a calibrated stereo camera with small field of view overlap

Abstract

In this paper we present a new minimal solver for the relative pose of a calibrated stereo camera. It is based on the observation that a feature visible in all cameras constrains the relative pose of the second stereo camera to be on a sphere around the feature which has a known position relative to the first stereo camera pose due to its triangulation. The constraint leaves three degrees of freedom, two for the location of the second camera on the sphere and the third for rotation in the plane tangent to the sphere. We use three temporal 2D correspondences, two correspondences from the left (or right) camera and one correspondence from the other camera to solve for these three remaining degrees of freedom. This approach is amenable to stereo pairs having a small overlap in their views. We present an efficient solution of this novel relative pose problem, theoretically derive how to use our new solver with two classes of measurements in RANSAC, evaluate its performance given noise and outliers and demonstrate its use in a real-time structure from motion system.

2-Entity RANSAC for robust visual localization in changing environment

“…If it is a point match, we perform 2P, if it is a line match, we perform 1P1L. There are some works discuss about sampling points from multiple combinations of views [28] [29], while we are focusing on sampling from multiple types of features (points and lines). In this section, we derive the success probability of the three sampling strategies to provide insights of sampling in 2-entity RANSAC.…”

Automatic Camera Calibration Using Multiple Sets of Pairwise Correspondences

“…In the case of the camera network having just two nodes the extrinsic calibration problem from pairwise correspondences can be potentially solved using methods developed for visual odometry using stereo cameras. There is a minimal solution for the relative pose between stereo pairs using 6 pairwise correspondences [16] that estimates an up-to scale relative pose solution using 5 correspondences with one camera [25] and solves the scale factor with an additional correspondence from another camera. A non-minimal solution using 10 correspondences was also proposed for the case of any arbitrary combination of correspondences between the 4 views of two stereo rigs [17].…”

“…This problem is only briefly addressed in [16] when estimating the relative pose between stereo rigs. In that problem RANSAC must independently select one sample from 3 datasets containing 2-view, 3-view, and 4-view correspondences.…”

“…It is shown that the number of RANSAC iterations must be computed in a different way when the different datasets have different inlier ratios, and also that different types of correspondences should be weighted differently on the cost function. However, the observations made in [16] can only be directly extended to problems where we know exactly how many samples are selected in each dataset. In our calibration problem this might not be the case, since there are different combinations of pairwise correspondences among different cameras that can generate a solution.…”

“…Thus, we propose a modified RANSAC version specifically designed to simultaneously sample multiple datasets. The usefulness of such RANSAC extension goes beyond the calibration problem at hands and can benefit other algorithms such as Structure-from-Motion (SfM) using stereo cameras [16], [17], multi-view camera rigs [18], [19], or a mixture of point and plane correspondences [20], [21]. The contributions in this paper can be summarised as follows:…”