Robust Visual-Inertial Odometry in Dynamic Environments Using Semantic Segmentation for Feature Selection

Abstract

Abstract. Camera based navigation in dynamic environments with high content of moving objects is challenging. Keypoint-based localization methods need to reliably reject features that do not belong to the static background. Here, traditional statistical methods for outlier rejection quickly reach their limits. A common approach is the combination with an inertial measurement unit for visual-inertial odometry. Also, deep learning based semantic segmentation was recently successfully applied in camera based localization to identify features on common objects. In this work, we study the application of mask-based feature selection based on semantic segmentation for robust localization in high dynamic environments. We focus on visual-inertial odometry, but similarly investigate a state-of-the-art pure vision-based method as baseline. For a versatile evaluation, we use challenging self-recorded datasets based on different sensor systems. This includes a combined dataset of a real world system and its synthetic clone with a large number of humans for in-depth analysis. We further deploy large-scale datasets from pedestrian navigation in a mall with escalator scenes and vehicle navigation during the day and at night. Our results show that visual-inertial odometry performs generally well in dynamic environments itself, but also shows significant failures in challenging scenes, which are prevented by using the segmentation aid.

Seamless Navigation, 3D Reconstruction, Thermographic and Semantic Mapping for Building Inspection

“…IPS localization and 3D reconstruction capability and robustness have been evaluated in various environments in [9,43]. It is suitable for large-scale 3D reconstruction, as is required, for example, in mining inspection [44], as well as in other inspection tasks in partially tight or generally difficult environments such as tunnels, ships, and building interiors.…”