Keyframe-based visual–inertial odometry using nonlinear optimization

Abstract

Combining visual and inertial measurements has become popular in mobile robotics, since the two sensing modalities offer complementary characteristics that make them the ideal choice for accurate Visual-Inertial Odometry or Simultaneous Localization and Mapping (SLAM). While historically the problem has been addressed with filtering, advancements in visual estimation suggest that non-linear optimization offers superior accuracy, while still tractable in complexity thanks to the sparsity of the underlying problem. Taking inspiration from these findings, we formulate a rigorously probabilistic cost function that combines reprojection errors of landmarks and inertial terms. The problem is kept tractable and thus ensuring real-time operation by limiting the optimization to a bounded window of keyframes through marginalization. Keyframes may be spaced in time by arbitrary intervals, while still related by linearized inertial terms. We present evaluation results on complementary datasets recorded with our custom-built stereo visual-inertial hardware that accurately synchronizes accelerometer and gyroscope measurements with imagery. A comparison of both a stereo and monocular version of our algorithm with and without online extrinsics estimation is shown with respect to ground truth. Furthermore, we compare the performance to an implementation of a state-of-the-art stochasic cloning sliding-window filter. This competititve reference implementation performs tightly-coupled filtering-based visual-inertial odometry. While our approach declaredly demands more computation, we show its superior performance in terms of accuracy.

Multi-modal and inertial sensor solutions for navigation-type factor graphs

“…Our work presents a theoretical development of inertial preintegral bias compensation models and best relates to work by [143], [103] and [137]. Our work extends that of [103] and [143] by presenting a continuous time analytical derivation of a Taylor expansion of the sensor error terms manifold inside each odometry likelihood function.…”

“…Recently Leutenegger et al [137] published work on a visual inertial SLAM solution, which does indeed estimate bias terms. Their work presents an excellent overview of visual inertial systems, but does not present complete analytical models for compensated interpose inertial constraints; their work does mention the need for compensation Jacobians, but are not presented.…”

Loop-Closure Detection in Urban Scenes for Autonomous Robot Navigation

“…As binary features are computationally drastically more efficient than their floating point counterparts (e.g. SURF), they are most commonly used during SLAM [11,14]. As a result, re-using them for place recognition promises to eliminate unnecessary computational effort, however, the robustness of place recognition systems based on binary features to common scene variations is limited.…”

“…In this work, we use the open-source keyframe-based VI SLAM algorithm OKVIS [11], which estimates the trajectory of the robot considering a limited window of past poses, and as a result has no loop-closure detection or correction scheme. OKVIS provides in real-time, the current robot pose P and a 3D map comprising of the estimated locations of 3D visual landmarks extracted from the image feed.…”

Cooperative visual-inertial sensor fusion: Fundamental equations

“…The problem of fusing visual and inertial data for single robots has been extensively investigated in the past [1], [3], [9], [15], [22]. Recently, this sensor fusion problem has been successfully addressed by enforcing observability constraints [7], [11], and by using optimization-based approaches [4], [10], [14], [21], [25], [32], [33]. These optimization methods outperform filter-based algorithms in terms of accuracy due to their capability of relinearizing past states.…”