Dynamic body VSLAM with semantic constraints

Abstract

Abstract-Image based reconstruction of urban environments is a challenging problem that deals with optimization of large number of variables, and has several sources of errors like the presence of dynamic objects. Since most large scale approaches make the assumption of observing static scenes, dynamic objects are relegated to the noise modeling section of such systems. This is an approach of convenience since the RANSAC based framework used to compute most multiview geometric quantities for static scenes naturally confine dynamic objects to the class of outlier measurements. However, reconstructing dynamic objects along with the static environment helps us get a complete picture of an urban environment. Such understanding can then be used for important robotic tasks like path planning for autonomous navigation, obstacle tracking and avoidance, and other areas.In this paper, we propose a system for robust SLAM that works in both static and dynamic environments. To overcome the challenge of dynamic objects in the scene, we propose a new model to incorporate semantic constraints into the reconstruction algorithm. While some of these constraints are based on multi-layered dense CRFs trained over appearance as well as motion cues, other proposed constraints can be expressed as additional terms in the bundle adjustment optimization process that does iterative refinement of 3D structure and camera / object motion trajectories. We show results on the challenging KITTI urban dataset for accuracy of motion segmentation and reconstruction of the trajectory and shape of moving objects relative to ground truth. We are able to show average relative error reduction by a significant amount for moving object trajectory reconstruction relative to state-ofthe-art methods like VISO 2[16], as well as standard bundle adjustment algorithms.

Utilizing semantic visual landmarks for precise vehicle navigation

“…There are a few methods which incorporate semantic information into SLAM systems for vehicle navigation. For example, Reddy et al [14] use a multi-layer conditional random field (CRF) framework to perform motion segmentation and object class labeling. It improves localization performance by only mapping stationary objects in the environment and excluding dynamic objects from the scene.…”

Robust Dense Mapping for Large-Scale Dynamic Environments

“…Reddy et al [22] describe a stereo-based algorithm for robust SLAM that works in both static and dynamic environments. They separate the static background from the moving objects by minimizing a joint CRF-based semantic motion segmentation energy function.…”

“…The sensors typically used in 3D mapping include depth [19], monocular [30], and stereo [22] cameras. While depth cameras can readily provide high-quality depth measurements, their usage is limited to indoor environments.…”

A Unified Framework for Mutual Improvement of SLAM and Semantic Segmentation

“…For example, [7] computed the likelihood of a moving object based on a motion metric computed from optical flow and then segment the moving objects. [8] further extended it to handle stereo image sequences. Recently, researchers have shifted their focus to using deep neural network to do the segmentation to remove outliers for accurate pose estimation.…”