Air-ground localization and map augmentation using monocular dense reconstruction

Abstract

We propose a new method for the localization of a Micro Aerial Vehicle (MAV) with respect to a ground robot. We solve the problem of registering the 3D maps computed by the robots using different sensors: a dense 3D reconstruction from the MAV monocular camera is aligned with the map computed from the depth sensor on the ground robot. Once aligned, the dense reconstruction from the MAV is used to augment the map computed by the ground robot, by extending it with the information conveyed by the aerial views. The overall approach is novel, as it builds on recent developments in live dense reconstruction from moving cameras to address the problem of air-ground localization. The core of our contribution is constituted by a novel algorithm integrating dense reconstructions from monocular views, Monte Carlo localization, and an iterative pose refinement. In spite of the radically different vantage points from which the maps are acquired, the proposed method achieves high accuracy whereas appearance-based, state-of-the-art approaches fail. Experimental validation in indoor and outdoor scenarios reported an accuracy in position estimation of 0.08 meters and real time performance. This demonstrates that our new approach effectively overcomes the limitations imposed by the difference in sensors and vantage points that negatively affect previous techniques relying on matching visual features. Air-Ground Localization and Map Augmentation Using Monocular Dense ReconstructionChristian Forster, Matia Pizzoli, Davide ScaramuzzaAbstract-We propose a new method for the localization of a Micro Aerial Vehicle (MAV) with respect to a ground robot. We solve the problem of registering the 3D maps computed by the robots using different sensors: a dense 3D reconstruction from the MAV monocular camera is aligned with the map computed from the depth sensor on the ground robot. Once aligned, the dense reconstruction from the MAV is used to augment the map computed by the ground robot, by extending it with the information conveyed by the aerial views. The overall approach is novel, as it builds on recent developments in live dense reconstruction from moving cameras to address the problem of airground localization. The core of our contribution is constituted by a novel algorithm integrating dense reconstructions from monocular views, Monte Carlo localization, and an iterative pose refinement. In spite of the radically different vantage points from which the maps are acquired, the proposed method achieves high accuracy whereas appearance-based, state-of-theart approaches fail. Experimental validation in indoor and outdoor scenarios reported an accuracy in position estimation of 0.08 meters and real time performance. This demonstrates that our new approach effectively overcomes the limitations imposed by the difference in sensors and vantage points that negatively affect previous techniques relying on matching visual features.

Collaborative navigation for flying and walking robots

“…Hence, the system is readily extendable to groups of more than two robots. The approaches of Michael et al [10] and Forster et al [11] are conceptually similar to ours, yet based on geometric registration, which is known to be sensitive to initial guesses. We experimentally demonstrate that our feature-based visual localization and mapping framework enables reliable operation of both aerial and ground robots using a single map -despite the strongly varying viewpoints of the different agents, and without requiring prior information about the relative poses of the robots.…”

“…Moreover, unlike [10] and [11], we integrate localization and mapping with a complete system for autonomous navigation of the ground robot, comprising collaborative terrain mapping using both cameras and laser scanners (multi-modal mapping), as well as traversability estimation, path planning, and motion control. To this end, we also integrate the high-level localization and mapping framework into our high-bandwidth but drift-affected Extended Kalman Filter-based (EKF) state estimation fusing kinematics and inertial measurements.…”

Collaborative localization of aerial and ground robots through elevation maps

“…For the method to succeed, no walls or ceilings are necessary, which is of high importance for outdoor applications. After an initialization period of a few seconds, tracking of the ground robot is performed in realtime using a particle filter [12], [15].…”

“…Similar to the work of Forster et al [15] on the localization of a quadrotor within a given map of a ground robot, we use 2.5D elevation maps generated by the robots to compute the relative position. We extend the work in [15] by evaluating different similarity metrics.…”

From Video Games Multiple Cameras to Multi-robot Teleoperation in Disaster Scenarios

“…In the past few years a wide range of applications using Unnamed Aerial Vehicles (UAV's) as a tool to increase the visual information available to ground robots, have been developed [5], [6], [7], [8], [9]. Nevertheless, there are some drawbacks when using UAV's: Firstly, due to battery restrictions they cannot operate in long term missions, also, the topdown view provided may not be helpful in cluttered scenarios or when a low height roof is present, finally it is necessary to solve their Guidance, Navigation and Control difficulties, which may be very complex in disaster scenarios.…”