Monocular Semantic Mapping Based on 3D Cuboids Tracking

Abstract

Semantic mapping based on information of objects has become a crucial component for the surrounding comprehension and the more robust navigation. In this paper, we propose a system for simultaneous localization and mapping (SLAM) that combines multiple objects tracking and factor graph optimization with semantically meaningful landmarks to achieve accurate monocular semantic mapping. Firstly, the process of object recognition uses a vanishing point sampling-based approach to efficiently infer the class and position of object landmarks from 2D bounding box object detection. Secondly, The semantic frontend utilizes local matching-based data association to track raw cuboid proposals. It can provide semantic constraints to reduce cuboid scale drift and improve its position estimation. Finally, we present a multi-view factor graph optimization which can use motion modal of the camera to optimize stable cuboids. The semantic mapping experiments on our own built virtual scene show better accuracy and robustness over existing approaches. We evaluate the effectiveness of our approach on public KITTI datasets and a real scene.Index Terms-visual simultaneous localization and mapping (SLAM), 3D semantic map, factor graph optimization.

Tightly-Coupled SLAM Integrating LiDAR and INS for Unmanned Vehicle Navigation in Campus Environments

“…SLAM++ proposed by Renato et al [8] in 2013 can be regarded as the earliest SLAM algorithm using semantic information. In recent years, a series of LLOAM [9], SegMap [10], CubeSLAM [11,12] based on point cloud segmentation, SuMa++ [13] based on semantic information, ApriISAM [14] and deep learn-based GEN-SLAM [15] have appeared in various peaks [16]. Gmapping is based on 2D liDAR and LOAM is based on 3D liDAR [17,18].…”