Dynamic Object Tracking and Masking for Visual SLAM

Abstract

In dynamic environments, performance of visual SLAM techniques can be impaired by visual features taken from moving objects. One solution is to identify those objects so that their visual features can be removed for localization and mapping. This paper presents a simple and fast pipeline that uses deep neural networks, extended Kalman filters and visual SLAM to improve both localization and mapping in dynamic environments (around 14 fps on a GTX 1080). Results on the dynamic sequences from the TUM dataset using RTAB-Map as visual SLAM suggest that the approach achieves similar localization performance compared to other state-of-the-art methods, while also providing the position of the tracked dynamic objects, a 3D map free of those dynamic objects, better loop closure detection with the whole pipeline able to run on a robot moving at moderate speed.

Semi-Supervised 3d Object Detection Via Adaptive Pseudo-Labeling

“…3D object detection [1,2,3] is an important task in computer vision and has various applications such as autonomous driving [4] and robotics [5,6]. The goal of this task is to estimate the categories and corresponding 3D bounding boxes of all targets in the scene.…”

Real-Time Metric-Semantic Visual Slam for Dynamic and Changing Environments

“…DOTMask [52], proposed by Vincent et al, uses instance segmentation to obtain the pixel-wise information of the objects in the image, and an Extended Kalman Filter to track these objects. Their aim was to provide a faster SLAM system in exchange of a lower accuracy, in comparison with DynaSLAM, for example.…”

“…The proposed method has a robust keypoint classification algorithm that filters a priori dynamic objects and uses an Extended Kalman Filter to track movable objects in the scene. This resulted in a visual SLAM system for highly dynamic environments that runs faster than DOT-Mask [52] and the method of Ji et al [53], with an accuracy similar to DynaSLAM [30] and SaD-SLAM [32]. Furthermore, the problem of feature depletion caused by filtering features from the background in the bounding boxes is solved with a fast and reliable method, using statistical data of the depth in each bounding box.…”

“…The system was compared to several others discussed in Chapter 1. The comparison includes featurebased methods designed for static environments, such as ORB-SLAM3 [3], direct methods designed for dynamic environments, such as StaticFusion [35] and ReFusion [36], and feature-based methods designed for dynamic environments, for instance DynaSLAM [30], DS-SLAM [31], SaD-SLAM [32], and DOT-Mask [52].…”

“…For instance, SaD-SLAM [32] is offline, and DynaSLAM [30] runs at less than 1 FPS. DOT-Mask [52], even using GPU, only achieves 8 FPS.…”