Parallel, real-time monocular visual odometry

Abstract

Abstract-We present a real-time, accurate, large-scale monocular visual odometry system for real-world autonomous outdoor driving applications. The key contributions of our work are a series of architectural innovations that address the challenge of robust multithreading even for scenes with large motions and rapidly changing imagery. Our design is extensible for three or more parallel CPU threads. The system uses 3D-2D correspondences for robust pose estimation across all threads, followed by local bundle adjustment in the primary thread. In contrast to prior work, epipolar search operates in parallel in other threads to generate new 3D points at every frame. This significantly boosts robustness and accuracy, since only extensively validated 3D points with long tracks are inserted at keyframes. Fast-moving vehicles also necessitate immediate global bundle adjustment, which is triggered by our novel keyframe design in parallel with pose estimation in a thread-safe architecture. To handle inevitable tracking failures, a recovery method is provided. Scale drift is corrected only occasionally, using a novel mechanism that detects (rather than assumes) local planarity of the road by combining information from triangulated 3D points and the inter-image planar homography. Our system is optimized to output pose within 50 ms in the worst case, while average case operation is over 30 fps. Evaluations are presented on the challenging KITTI dataset for autonomous driving, where we achieve better rotation and translation accuracy than other state-of-the-art systems.

Robust stereo visual odometry from monocular techniques

“…As argued by Song et al this is in general not practical for automotive applications [1]. Alternative approaches keeping track solely of recent map information have become more popular in recent years [1], [4], [5], [5], [17].…”

“…The top performing visual-only systems to date are all stereo based papers [4], [17], [13], [5]. An exception to this rule is given by the work of Song et al which is among the best scoring systems despite relying on monocular data [1]. This system is a feature based, local method using bundleadjustment.…”

“…These matches are then used to compute the egomotion by the means of essential matrix estimation [11], PNP [4], [1] or the trifocal tensor [12]. Direct methods on the other hand operate on the whole images thus producing a dense or semi-dense motion field [13], [14], [15], [16].…”

“…Monocular approaches make use solely of the information provided by a single moving camera, thus information on the scene and the motion can only be recovered up to a global scaling factor without assumptions on the scene [1], [25]. Stereo systems, on the other hand, can rely on the known transformation between the two cameras to extract the metric scale and on the redundancy of the visual data to improve stability and robustness [4], [25], [17], [12], [5], [13].…”

“…In particular we use motion model predicted tracking by matching not dissimilar to the scheme proposed by Song et al [1], delayed outlier identification [2], long optimization windows and robust iterative triangulation.…”