Online Self-Calibration for Visual-Inertial Navigation Systems: Models, Analysis and Degeneracy

Abstract

In this paper, we study in-depth the problem of online self-calibration for robust and accurate visual-inertial state estimation. In particular, we first perform a complete observability analysis for visual-inertial navigation systems (VINS) with full calibration of sensing parameters, including IMU and camera intrinsics and IMU-camera spatial-temporal extrinsic calibration, along with readout time of rolling shutter (RS) cameras (if used). We investigate different inertial model variants containing IMU intrinsic parameters that encompass most commonly used models for low-cost inertial sensors. With these models, the state transition matrix and visual measurement Jacobians are analytically derived and the observability analysis of linearized VINS with full sensor calibration is performed. The analysis results prove that, as intuitively assumed in the literature, VINS with full sensor calibration has four unobservable directions, corresponding to the system's global yaw and translation, while all sensor calibration parameters are observable given fully-excited 6-axis motion. Moreover, we, for the first time, identify primitive degenerate motions for IMU and camera intrinsic calibration, which, when combined, may produce complex degenerate motions. This result holds true for the different inertial model variants investigated in this work and has significant impacts on practical applications of online self-calibration to many robotic platforms. Extensive Monte-Carlo simulations and real-world experiments are performed to validate both the observability analysis and identified degenerate motions, showing that online self-calibration improves system accuracy and robustness to calibration inaccuracies. We compare the proposed online self-calibration on commonly-used IMUs against the state-of-art offline calibration toolbox Kalibr, and show that the proposed system achieves better consistency and repeatability. As sensor calibration plays an important role in making real VINS work well in practice, based on our analysis and experimental evaluations, we also provide practical guidelines for how to perform online IMU-camera sensor self-calibration.

NOCT: Nonlinear Observability with Constraints and Time Offset

“…Using the observability matrix built with piece-wise constant transition functions, Li and Mourikis [10] improved consistency of a filter-based VIO algorithm by maintaining observability properties of VIO. Recently, [23] analyzed the unobservable directions of a linearized VIO system with full self-calibration under motion constraints. Li and Stueckler [9] incorporated constraints due to velocity controls and planar motion as observations into the VIO system and analyzed observability of state variables by using the linearized system model.…”

“…By manually identifying the unobservable directions, this approach is likely to miss some unobservable directions. For instance, consider the observability properties obtained in [23] based on a linearized VIO model, the camera-IMU system would have a different number of unobservable calibration parameters when the axis labels of the IMU are swapped (e.g., x-y-z to z-x-y), indicating that some unobservable parameters are missed out by manual checking.…”

“…For a camera-IMU system, Yang et al . [23] analyzed cases where the time offset is unobservable by using a linearized system model. Recently, in [4], degenerate cases for time offset estimation were identified for a nonlinear system of affine inputs, encompassing cases reported in [24,23].…”

“…[23] analyzed cases where the time offset is unobservable by using a linearized system model. Recently, in [4], degenerate cases for time offset estimation were identified for a nonlinear system of affine inputs, encompassing cases reported in [24,23]. This paper provides additional simulation results to supplement the findings of [4].…”

“…Using the procedures presented in Section 4.2, we incorporate these three types of constraints separately into the unconstrained VIO system (20)-( 22) and convert each of the three specific systems into the standard form. For the motion of constant local acceleration (23), we use the procedure for System 5. For the single-axis rotation constraint (24) and the pure translation constraint (27), the procedure for System 4 is applied.…”