Exploiting Urban Scenes for Vision-aided Inertial Navigation

Abstract

Abstract-This paper addresses the problem of visual-inertial navigation when processing camera observations of both point and line features detected within a Manhattan world. First, we prove that the observations of: (i) a single point, and (ii) a single line of known direction perpendicular to gravity (e.g., a non-vertical structural line of a building), provide sufficient information for rendering all degrees of freedom of a visionaided inertial navigation system (VINS) observable, up to global translations. Next, we examine the observability properties of the linearized system employed by an extended Kalman filter (EKF) for processing line observations of known direction, and show that the rank of the corresponding observability matrix erroneously increases. To address this problem, we introduce an elegant modification which enforces that the linearized EKF system has the correct number of unobservable directions, thus improving its consistency. Finally, we validate our findings experimentally in urban scenes and demonstrate the superior performance of the proposed VINS over alternative approaches.

StructVIO: Visual-Inertial Odometry With Structural Regularity of Man-Made Environments

“…Most existing methods explore only the partial information of structural regularity -they either use straight lines without considering their prior orientation, or use the prior orientation without putting lines as extra measurements for better estimation. A few of existing methods consider both aspects [11] [23]. In [11], the lines with prior orientation are named as structural lines and treated them as landmarks the same as point features for visual SLAM.…”

Using vanishing points to improve visual-inertial odometry

“…Related to our approach [7] proposed to use observations of lines with a known direction along with point feature measurements to remove the unobservability around yaw. In their method they directly observe image lines and assign them to a world direction using a Mahalanobis check, for which they need to initially align the system's yaw with one of the building's VPs.…”

“…Under several motion profiles this will lead to a loss in accuracy for yaw [6]. This may lead to accumulation of errors along these unobservable states which will tend to grow linearly with time [7].…”

Hierarchical Semantic Labeling for Task-Relevant RGB-D Perception

“…There is also a huge body of works using vision algorithms to help perform different robotic tasks [13,38,16,30], such as object grasping [42,11,29], navigation [6,26], trajectory control [44], and activity anticipation [23]. Many works focused on improving SLAM techniques to better depict an environment for planning and navigation [34,28], such as incremental smoothing and mapping using the Bayes Tree [21], real-time visual SLAM over large-scale environments [46], and object level SLAM [40].…”