Predicting Unobserved Space for Planning via Depth Map Augmentation

Abstract

Safe and efficient path planning is crucial for autonomous mobile robots. A prerequisite for path planning is to have a comprehensive understanding of the 3D structure of the robot's environment. On Micro Air Vehicles (MAVs) this is commonly achieved using low-cost sensors, such as stereo or RGB-D cameras. These sensors may fail to provide depth measurements in textureless or IR-absorbing areas and have limited effective range. In path planning, this results in inefficient trajectories or failure to recognize a feasible path to the goal, hence significantly impairing the robot's mobility. Recent advances in deep learning enables us to exploit prior experience about the shape of the world and hence to infer complete depth maps from color images and additional sparse depth measurements. In this work, we present an augmented planning system and investigate the effects of employing stateof-the-art depth completion techniques, specifically trained to augment sparse depth maps originating from RGB-D sensors, semi-dense methods and stereo matchers. We extensively evaluate our approach in online path planning experiments based on simulated data, as well as global path planning experiments based on real world Micro Air Vehicle (MAV) data. We show that our augmented system, provided with only sparse depth perception, can reach on-par performance to ground truth depth input in simulated online planning experiments. On real world MAV data the augmented system demonstrates superior performance compared to a planner based on very dense RGB-D depth maps.

A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers

“…Recent related work, however, addresses the problem of scene completion and occupancy anticipation from a DL perspective. Fehr et al [231] use a neural network to augment the measurements of a depth sensor and Ramakrishnan et al [232] directly predict augmented OG maps beyond the sensor's field-of-view using auto-encoders (AE). Rather than using raw sensor measurements, Katyal et al [233] and Hayoun et al [234] extend an input OG map beyond the lineof-sight also using AE.…”

A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers

“…Recent related work, however, addresses the problem of scene completion and occupancy anticipation from a DL perspective. Fehr et al [221] use a neural network to augment the measurements of a depth sensor and Ramakrishnan et al [222] directly predict augmented OG maps beyond the sensor's field-of-view using auto-encoders (AE). Rather than using raw sensor measurements, Katyal et al [223] and Hayoun et al [224] extend an input OG map beyond the lineof-sight also using AE.…”

3D Lidar Reconstruction with Probabilistic Depth Completion for Robotic Navigation

“…While many works aim to address the problem of depth completion on a per-image basis, only a few investigate the use of the dense depth output for applications such as reconstruction and navigation. Fehr et al [14] proposed a planning system that predicts dense depth images using a CNN [2] and uses them as the input for the Voxblox mapping system [15]. Depth estimation has been used in visual SLAM systems to recover metric scale [16,17], and to complete sparse visual features [18,19].…”