Fury of the Swarm: Efficient and Very Accurate Triangulation for Multi-view Scene Reconstruction

Abstract

This paper presents a novel framework for practical and accurate N -view triangulation of scene points. The algorithm is based on applying swarm optimization inside a robustly-computed bounding box, using an angular errorbased L 1 cost function which is more robust to outliers and less susceptible to local minima than cost functions such as L 2 on reprojection error. Extensive testing on synthetic data with ground-truth has determined an accurate position over 99.9% of the time, on thousands of camera configurations with varying degrees of feature tracking errors. Opposed to existing polynomial methods developed for a small number of cameras, the proposed algorithm is at best linear in the number of cameras and does not suffer from inaccuracies inherent in solving high-order polynomials or Gröbner bases. In the specific case of three views, there is a two to three order of magnitude performance increase with respect to such methods. Results are provided to highlight performance for arbitrary camera configurations, numbers of cameras and under noise, which has not been previously achieved in the triangulation literature. Results on real data also prove that reprojection error is improved with respect to other methods.

Depth data assisted structure-from-motion parameter optimization and feature track correction

“…The first is a general cost function for parameter optimization. The cost function evaluates the quality of a reconstruction based upon an angular error metric, presented by Recker et al [23], and depth data estimates. The second, as opposed to traditional bundle adjustment, in the event of feature tracking errors, is a corrective routine to detect and correct inaccurate feature tracks, based upon depth data as opposed to traditional epipolar constraints.…”

“…The cost function takes into account two error measures: first, the angular error between each computed 3D scene point and its corresponding feature track location [23], and second, the difference between the sensor depth value and its computed estimate. Figure 1 contains a visual depiction of the cost function.…”

“…Here C i is a single camera (and its associated data) and s j is a 3D scene point. E angular is the angular error used by Recker et al [23] and is rewritten in Eq. 5 in its singular form (for one camera and scene point).…”

“…Note that C i,c is the camera center of projection (camera position), C i,P + is the right pseudo-inverse of the camera projection matrix, and k j is the feature track location of s j in C i . Also, note that v and w are normalized to have unit length, as in the original formulation [23].…”

“…Despite these problems, a simple yet effective approach to obtaining insight into a function's topology is to perform a scalar field analysis similar to that employed by Recker et al [23], as shown in Fig. 2.…”