SLAMANTIC - Leveraging Semantics to Improve VSLAM in Dynamic Environments

Schörghuber, Matthias; Steininger, Daniel; Cabon, Yohann; Humenberger, Martin; Gelautz, Margrit

doi:10.1109/iccvw.2019.00468

Cited by 27 publications

(26 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The semantic segmentation network can be implemented inside a SLAM framework by various approaches, as in other methods like [36] and [16]. The output of this network, as stated before, is used for filtering out the undesired (dynamic) keypoints.…”

Section: A Preprocessingmentioning

confidence: 99%

SRVIO: Super Robust Visual Inertial Odometry for dynamic environments and challenging Loop-closure conditions

Samadzadeh¹,

Nickabadi²

2022

Preprint

View full text Add to dashboard Cite

The visual localization or odometry problem is a well-known challenge in the field of autonomous robots and cars. Traditionally, this problem can ba tackled with the help of expensive sensors such as lidars. Nowadays, the leading research is on robust localization using economic sensors, such as cameras and IMUs. The geometric methods based on these sensors are pretty good in normal conditions withstable lighting and no dynamic objects. These methods suffer from significant loss and divergence in such challenging environments. The scientists came to use deep neural networks (DNNs) as the savior to mitigate this problem. The main idea behind using DNNs was to better understand the problem inside the data and overcome complex conditions (such as a dynamic object in front of the camera, extreme lighting conditions, keeping the track at high speeds, etc.) The prior endto-end DNN methods are able to overcome some of the mentioned challenges. However, no general and robust framework for all of these scenarios is available. In this paper, we have combined geometric and DNN based methods to have the pros of geometric SLAM frameworks and overcome the remaining challenges with the DNNs help. To do this, we have modified the Vins-Mono framework (the most robust and accurate framework till now) and we were able to achieve state-of-the-art results on TUM-Dynamic, TUM-VI, ADVIO and EuRoC datasets compared to geometric and endto-end DNN based SLAMs. Our proposed framework was also able to achieve acceptable results on extreme simulated cases resembling the challenges mentioned earlier easy. 1

show abstract

Section: A Preprocessingmentioning

confidence: 99%

SRVIO: Super Robust Visual Inertial Odometry for dynamic environments and challenging Loop-closure conditions

Samadzadeh¹,

Nickabadi²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We donate δ as the threshold to determine whether the feature points are dynamic or not, and the way to judge it is to calculate the distance k i of each feature point by using formula (8). If k i > δ, the feature point will be labeled as dynamic Otherwise, it will be labeled as static.…”

Section: ) Dynamic Features Detectionmentioning

confidence: 99%

“…Identifying and excluding feature points on dynamic objects is an effective way to eliminates the impact of dynamic environments on the system [7]. Recent studies [8], [9] have shown that feature points can be effectively classified based on the results of semantic segmentation. And then, only feature points located on static objects are allowed to participate in subsequent calculations.…”

Section: Introductionmentioning

confidence: 99%

Real-Time Visual-Inertial Localization Using Semantic Segmentation Towards Dynamic Environments

2020

View full text Add to dashboard Cite

Simultaneous localization and mapping(SLAM), focusing on addressing the joint estimation problem of self-localization and scene mapping, has been widely used in many applications such as mobile robot, drone, and augmented reality(AR). However, traditional state-of-the-art SLAM approaches are typically designed under the static-world assumption and prone to be degraded by moving objects when running in dynamic scenes. This paper presents a novel semantic visual-inertial SLAM system for dynamic environments that, building on VINS-Mono, performs real-time trajectory estimation by utilizing the pixelwise results of semantic segmentation. We integrate the feature tracking and extraction framework into the front-end of the SLAM system, which could make full use of the time waiting for the completion of the semantic segmentation module, to effectively track the feature points on subsequent images from the camera. In this way, the system can track feature points stably even in high-speed movement. We also construct the dynamic feature detection module that combines the pixel-wise semantic segmentation results and the multi-view geometric constraints to exclude dynamic feature points. We evaluate our system in public datasets, including dynamic indoor scenes and outdoor scenes. Several experiments demonstrate that our system could achieve higher localization accuracy and robustness than state-of-the-art SLAM systems in challenging environments.INDEX TERMS Simultaneous localization and mapping, dynamic environment, semantic, visual-inertial system.

show abstract

“…(Wang et al, 2019) simultaneously improved SLAM and semantic segmentation by distinguishing between features on moving, potentially moving and on the static background for SLAM and using the 3D pose information to refine the segmentation. (Schorghuber et al, 2019) distinguished between similar object states in a dynamic fashion, using a continuously updated confidence factor. In contrast, we decided to use the basic masking approach, since many slowly moving objects are only observed for a short time in our handheld datasets.…”

Section: Related Workmentioning

confidence: 99%

Robust Visual-Inertial Odometry in Dynamic Environments Using Semantic Segmentation for Feature Selection

Irmisch

Baumbach

Ernst

2020

ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. Camera based navigation in dynamic environments with high content of moving objects is challenging. Keypoint-based localization methods need to reliably reject features that do not belong to the static background. Here, traditional statistical methods for outlier rejection quickly reach their limits. A common approach is the combination with an inertial measurement unit for visual-inertial odometry. Also, deep learning based semantic segmentation was recently successfully applied in camera based localization to identify features on common objects. In this work, we study the application of mask-based feature selection based on semantic segmentation for robust localization in high dynamic environments. We focus on visual-inertial odometry, but similarly investigate a state-of-the-art pure vision-based method as baseline. For a versatile evaluation, we use challenging self-recorded datasets based on different sensor systems. This includes a combined dataset of a real world system and its synthetic clone with a large number of humans for in-depth analysis. We further deploy large-scale datasets from pedestrian navigation in a mall with escalator scenes and vehicle navigation during the day and at night. Our results show that visual-inertial odometry performs generally well in dynamic environments itself, but also shows significant failures in challenging scenes, which are prevented by using the segmentation aid.

show abstract

SLAMANTIC - Leveraging Semantics to Improve VSLAM in Dynamic Environments

Cited by 27 publications

References 30 publications

SRVIO: Super Robust Visual Inertial Odometry for dynamic environments and challenging Loop-closure conditions

SRVIO: Super Robust Visual Inertial Odometry for dynamic environments and challenging Loop-closure conditions

Real-Time Visual-Inertial Localization Using Semantic Segmentation Towards Dynamic Environments

Robust Visual-Inertial Odometry in Dynamic Environments Using Semantic Segmentation for Feature Selection

Contact Info

Product

Resources

About