PL-VIO: Tightly-Coupled Monocular Visual–Inertial Odometry Using Point and Line Features

Abstract

To address the problem of estimating camera trajectory and to build a structural three-dimensional (3D) map based on inertial measurements and visual observations, this paper proposes point–line visual–inertial odometry (PL-VIO), a tightly-coupled monocular visual–inertial odometry system exploiting both point and line features. Compared with point features, lines provide significantly more geometrical structure information on the environment. To obtain both computation simplicity and representational compactness of a 3D spatial line, Plücker coordinates and orthonormal representation for the line are employed. To tightly and efficiently fuse the information from inertial measurement units (IMUs) and visual sensors, we optimize the states by minimizing a cost function which combines the pre-integrated IMU error term together with the point and line re-projection error terms in a sliding window optimization framework. The experiments evaluated on public datasets demonstrate that the PL-VIO method that combines point and line features outperforms several state-of-the-art VIO systems which use point features only.

Visual-Inertial Navigation: A Concise Review

“…The (right) invariant Kalman filter [56] was recently employed to improve filter consistency [25,57,58,59,60], as well as the (iterated) EKF that was also used for VINS in robocentric formulations [22,61,62,63]. On the other hand, in the EKF framework, different geometric features besides points have also been exploited to improve VINS performance, for example, line features used in [64,65,66,67,68] and plane features in [69,70,71,72]. In addition, the MSCKF-based VINS was also extended to use rolling-shutter cameras with inaccurate time synchronization [64,73], RGBD cameras [69,74], multiple cameras [53,75,76] and multiple IMUs [77].…”

Observability Analysis of Aided INS With Heterogeneous Features of Points, Lines, and Planes

“…As mentioned earlier, vision-aided INS (VINS) arguably is among the most popular localization methods in particular for resource-constrained sensor platforms such as mobile devices and micro aerial vehicles (MAVs) navigating in GPS-denied environments (e.g., see [26,27,10,28]). While most current VINS algorithms focus on using point features (e.g., [7,8,9,10]), line and plane features may not be blindly discarded in structured environments [29,30,31,32,33,34,35,36,24], in part because: (i) they are ubiquitous and compact in many urban or indoor environments (e.g., doors, walls, and stairs), (ii) they can be detected and tracked over a relatively long time period, and (iii) they are more robust in texture-less environments compared to point features.…”

“…Stacking (29), (32) and (33) yields the complete the measurement Jacobian of the plane measurement w.r.t. the state (1):…”

A New RGB-D SLAM Method with Moving Object Detection for Dynamic Indoor Scenes

“…Depending on the number of sensors, visual SLAM can be classified into monocular-camera- [8][9][10][11], stereo-camera- [12][13][14], and multiple-camera- [15,16] based versions. Monocular camera based visual SLAM cannot obtain scale information of the environment, so it is usually combined with inertial measurement units (IMUs) to obtain three dimensional (3D) information in the scene [17]. As for stereo-camera-and multiple-camera-based SLAM systems, they can obtain 3D coordinates of feature points by photogrammetry.…”