Probabilistic RGB-D odometry based on points, lines and planes under depth uncertainty

The emergence of modern RGB-D sensors had a significant impact in many application fields, including robotics, augmented reality (AR) and 3D scanning. They are low-cost, low-power and low-size alternatives to traditional range sensors such as LiDAR. Moreover, unlike RGB cameras, RGB-D sensors provide the additional depth information that removes the need of frame-by-frame triangulation for 3D scene reconstruction. These merits have made them very popular in mobile robotics and AR, where it is of great interest to estimate egomotion and 3D scene structure. Such spatial understanding can enable robots to navigate autonomously without collisions and allow users to insert virtual entities consistent with the image stream. In this chapter, we review common formulations of odometry and Simultaneous Localization and Mapping (known by its acronym SLAM) using RGB-D stream input. The two topics are closely related, as the former aims to track the incremental camera motion with respect to a local map of the scene, and the latter to jointly estimate the camera trajectory and the global map with consistency. In both cases, the standard approaches minimize a cost function using nonlinear optimization techniques. This chapter consists of three main parts: In the first part, we introduce the basic concept of odometry and SLAM and motivate the use of RGB-D sensors. We also give mathematical preliminaries relevant to most odometry and SLAM algorithms. In the second part, we detail the three main components of SLAM systems: camera pose tracking, scene mapping and loop closing. For each component, we describe different approaches proposed in the literature. In the final part, we provide a brief discussion on advanced research topics with the references to the state-of-the-art.

show abstract

“…Finally, [85] shows that it can be beneficial to use points, lines and planes all together in a joint optimization framework.…”

Section: Plane-based Methodsmentioning

confidence: 99%

RGB-D Odometry and SLAM

Civera

Lee

2019

Advances in Computer Vision and Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…Since different representations have their own advantages, some work has combined these models. For example, both Models 2 and 4 are used in [63], and in [64] Model 5 was used for plane fitting and Model 7 for formulating the cost function of plane matching. Our recent work [21] employed Models 2 and 3 for plane and its error state, respectively.…”

Section: Overview Of Plane Representationsmentioning

confidence: 99%

Observability Analysis of Aided INS With Heterogeneous Features of Points, Lines, and Planes

Yang

Huang

2019

IEEE Trans. Robot.

View full text Add to dashboard Cite

In this paper, we perform a thorough observability analysis for linearized inertial navigation systems (INS) aided by exteroceptive range and/or bearing sensors (such as cameras, LiDAR and sonars) with different geometric features (points, lines and planes). While the observability of vision-aided INS (VINS) with point features has been extensively studied in the literature, we analytically show that the general aided INS with point features preserves the same observability property -that is, 4 unobservable directions, corresponding to the global yaw and the global position of the sensor platform. We further prove that there are at least 5 (and 7) unobservable directions for the linearized aided INS with a single line (and plane) feature; and, for the first time, analytically derive the unobservable subspace for the case of multiple lines/planes. Building upon this, we examine the system observability of the linearized aided INS with different combinations of points, lines and planes, and show that, in general, the system preserves at least 4 unobservable directions, while if global measurements are available, as expected, some unobservable directions diminish. In particular, when using plane features, we propose to use a minimal, closest point (CP) representation; and we also study in-depth the effects of 5 degenerate motions identified on observability. To numerically validate our analysis, we develop and evaluate both EKFbased visual-inertial SLAM and visual-inertial odometry (VIO) using heterogeneous geometric features in Monte Carlo simulations. Related WorkAided INS is a classical research topic with significant body of literature [25] and has recently been reemerging in part due to the advancement of sensing and computing technologies. In this section, we briefly review the related literature closest to this work by focusing on the vision-aided INS. Aided INS with Points, Lines, and PlanesAs mentioned earlier, vision-aided INS (VINS) arguably is among the most popular localization methods in particular for resource-constrained sensor platforms such as mobile devices and micro aerial vehicles (MAVs) navigating in GPS-denied environments (e.g., see [26,27,10,28]). While most current VINS algorithms focus on using point features (e.g., [7,8,9,10]), line and plane features may not be blindly discarded in structured environments [29,30,31,32,33,34,35,36,24], in part because: (i) they are ubiquitous and compact in many urban or indoor environments (e.g., doors, walls, and stairs), (ii) they can be detected and tracked over a relatively long time period, and (iii) they are more robust in texture-less environments compared to point features.In the case of utilizing line features, Kottas et al. [29] represented the line with a quaternion and a distance scalar and studied the observability properties for linearized VINS with this line parameterization. Yu et al. [30] proposed a minimal four-parameter representation of line features for VIO using rolling-shutter cameras, while Zheng et al. [31] used two end points t...

show abstract

“…Several SLAM systems using plane primitives have emerged recently [1][2][3][4][5]. While, [1] showed how to compress dense reconstructed models by using plane segments, drift is reduced in [3,4] by using plane landmarks in a map optimization framework.…”

Section: Related Workmentioning

confidence: 99%

“…Man-made environments are predominantly made of planar surfaces, thus a recent trend in SLAM [1][2][3][4][5] for structured environments is to exploit plane primitives to reduce drift, reconstruct compact maps, and improve the robustness of visual odometry (VO). Such systems that are based on RGB-D cameras start typically by extracting plane segments from the depth map, by employing plane segmentation algorithms such as the one shown in Fig.…”

Section: Introductionmentioning

confidence: 99%

“…of applications, guaranteeing low latency is more important than obtaining precisely the exact segment boundaries. Secondly, we propose to use the cylinders primitives, given by CAPE, along with other features, for camera pose estimation by extending a probabilistic RGB-D Odometry framework [5] that already uses points, lines and planes. Following this framework, cylinder parameters are modelled probabilistically to account for uncertainty and pose is estimated by aligning cylinder axes.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fast Cylinder and Plane Extraction from Depth Cameras for Visual Odometry

Proença

Gao

2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

This paper presents CAPE, a method to extract planes and cylinder segments from organized point clouds, which processes 640×480 depth images on a single CPU core at an average of 300 Hz, by operating on a grid of planar cells. While, compared to state-of-the-art plane extraction, the latency of CAPE is more consistent and 4-10 times faster, depending on the scene, we also demonstrate empirically that applying CAPE to visual odometry can improve trajectory estimation on scenes made of cylindrical surfaces (e.g. tunnels), whereas using a plane extraction approach that is not curve-aware deteriorates performance on these scenes.To use these geometric primitives in visual odometry, we propose extending a probabilistic RGB-D odometry framework based on points, lines and planes to cylinder primitives. Following this framework, CAPE runs on fused depth maps and the parameters of cylinders are modelled probabilistically to account for uncertainty and weight accordingly the pose optimization residuals.

show abstract

Probabilistic RGB-D odometry based on points, lines and planes under depth uncertainty

Cited by 31 publications

References 42 publications

RGB-D Odometry and SLAM

RGB-D Odometry and SLAM

Observability Analysis of Aided INS With Heterogeneous Features of Points, Lines, and Planes

Fast Cylinder and Plane Extraction from Depth Cameras for Visual Odometry

Contact Info

Product

Resources

About