Semantic Monocular SLAM for Highly Dynamic Environments

Brasch, Nikolas; Božič, Aljaž; Lallemand, Joe; Tombari, Federico

doi:10.1109/iros.2018.8593828

Cited by 69 publications

(41 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motivated by the advances of deep learning and Convolutional Neural Networks (CNNs) for scene understanding, there have been many semantic SLAM techniques exploiting this information using cameras [5], [30], cameras + IMU data [4], stereo cameras [9], [14], [17], [32], [37], or RGB-D sensors [3], [18], [19], [25], [26], [28], [38]. Most of these approaches were only applied indoors and use either an object detector or a semantic segmentation of the camera image.…”

Section: Related Workmentioning

confidence: 99%

SuMa++: Efficient LiDAR-based Semantic SLAM

Chen

Milioto

Palazzolo

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

399

192

View full text Add to dashboard Cite

Reliable and accurate localization and mapping are key components of most autonomous systems. Besides geometric information about the mapped environment, the semantics plays an important role to enable intelligent navigation behaviors. In most realistic environments, this task is particularly complicated due to dynamics caused by moving objects, which can corrupt the mapping step or derail localization. In this paper, we propose an extension of a recently published surfelbased mapping approach exploiting three-dimensional laser range scans by integrating semantic information to facilitate the mapping process. The semantic information is efficiently extracted by a fully convolutional neural network and rendered on a spherical projection of the laser range data. This computed semantic segmentation results in point-wise labels for the whole scan, allowing us to build a semantically-enriched map with labeled surfels. This semantic map enables us to reliably filter moving objects, but also improve the projective scan matching via semantic constraints. Our experimental evaluation on challenging highways sequences from KITTI dataset with very few static structures and a large amount of moving cars shows the advantage of our semantic SLAM approach in comparison to a purely geometric, state-of-the-art approach.

show abstract

Section: Related Workmentioning

confidence: 99%

SuMa++: Efficient LiDAR-based Semantic SLAM

Chen

Milioto

Palazzolo

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

399

192

View full text Add to dashboard Cite

show abstract

“…Most of the existing literature in robotics models the dynamic targets as a single 3D point (Chojnacki and Indelman, 2018), or with a 3D pose and rely on lidar (Azim and Aycard, 2012), RGB-D cameras (Aldoma et al, 2013), monocular cameras (Li et al, 2018b), and visual–inertial sensing (Qiu et al, 2019). Related work also attempts to gain robustness against dynamic scenes by using an IMU (Hwangbo et al, 2009), masking portions of the scene corresponding to dynamic elements (Bescos et al, 2018; Brasch et al, 2018; Cui and Ma, 2019), or jointly tracking camera and dynamic objects (Bescos et al, 2020; Wang et al, 2007). To the best of the authors’ knowledge, the present article is the first work that attempts to perform visual–inertial SLAM, segment dense object models, estimate the 3D poses of known objects, and reconstruct and track dense human SMPL meshes.…”

Section: Related Workmentioning

confidence: 99%

Kimera: From SLAM to spatial perception with 3D dynamic scene graphs

Rosinol

Violette

Abate

et al. 2021

The International Journal of Robotics Research

141

View full text Add to dashboard Cite

Humans are able to form a complex mental model of the environment they move in. This mental model captures geometric and semantic aspects of the scene, describes the environment at multiple levels of abstractions (e.g., objects, rooms, buildings), includes static and dynamic entities and their relations (e.g., a person is in a room at a given time). In contrast, current robots’ internal representations still provide a partial and fragmented understanding of the environment, either in the form of a sparse or dense set of geometric primitives (e.g., points, lines, planes, and voxels), or as a collection of objects. This article attempts to reduce the gap between robot and human perception by introducing a novel representation, a 3D dynamic scene graph (DSG), that seamlessly captures metric and semantic aspects of a dynamic environment. A DSG is a layered graph where nodes represent spatial concepts at different levels of abstraction, and edges represent spatiotemporal relations among nodes. Our second contribution is Kimera, the first fully automatic method to build a DSG from visual–inertial data. Kimera includes accurate algorithms for visual–inertial simultaneous localization and mapping (SLAM), metric–semantic 3D reconstruction, object localization, human pose and shape estimation, and scene parsing. Our third contribution is a comprehensive evaluation of Kimera in real-life datasets and photo-realistic simulations, including a newly released dataset, uHumans2, which simulates a collection of crowded indoor and outdoor scenes. Our evaluation shows that Kimera achieves competitive performance in visual–inertial SLAM, estimates an accurate 3D metric–semantic mesh model in real-time, and builds a DSG of a complex indoor environment with tens of objects and humans in minutes. Our final contribution is to showcase how to use a DSG for real-time hierarchical semantic path-planning. The core modules in Kimera have been released open source.

show abstract

“…On the premise of the depth convergence of landmarks, the map is added. On this basis, a priori value is assigned to the static rate according to the output of the semantic segmentation network, and then the static rate of the landmark points is updated when new observation data is introduced to realize the smooth transition of the landmark points between dynamic and static [13]. In ICRA2018, Stenborg et al explored the stability of long-term positioning using semantic segmentation while participating in vehicle positioning projects.…”

Section: Related Workmentioning

confidence: 99%

Semantic Optimization of Feature-Based SLAM

Yin

Gao

Sun

2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

The purpose of this paper is to provide reasonable recommendation and removal of inappropriate information for SLAM (Simultaneous Localization and Mapping) technology based on feature method. The methodology is to propose a semantic recognition of environment objects in the natural scene through object detection, which is a kind of bag of word method in SLAM problem between the key frames and object level, the method of establishing key frames, and the relationship between the target object levels, through the practical signiﬁcance of the target object level to judge the merits of the target object level information, and then combined with key frames in the visual SLAM relations with relevant information, so as to get object level targets in each key frame and the relationship between the relevant information, so as to achieve through the object level semantic information to judge the merits of the key frames and screening, as well as to the key frames to judge the merits of the relevant information and screening purpose. The finding of the study is the above method can retain the information of high reliability and good stability for visual SLAM and process the key frames with poor reliability or low stability and the information related to key frames.

show abstract

Semantic Monocular SLAM for Highly Dynamic Environments

Cited by 69 publications

References 16 publications

SuMa++: Efficient LiDAR-based Semantic SLAM

SuMa++: Efficient LiDAR-based Semantic SLAM

Kimera: From SLAM to spatial perception with 3D dynamic scene graphs

Semantic Optimization of Feature-Based SLAM

Contact Info

Product

Resources

About