Towards evasive maneuvers with quadrotors using dynamic vision sensors

Abstract

We present a method to predict collisions with objects thrown at a quadrotor using a pair of dynamic vision sensors (DVS). Due to the micro-second temporal resolution of these sensors and the sparsity of their output, the object's trajectory can be estimated with minimal latency. Unlike standard cameras that send frames at a fixed frame rate, a DVS only transmits pixel-level brightness changes ("events") at the time they occur. Our method tracks spherical objects on the image plane using probabilistic trackers that are updated with each incoming event. The object's trajectory is estimated using an Extended Kalman Filter with a mixed state space that allows incorporation of both the object's dynamics and the measurement noise in the image plane. Using error-propagation techniques, we predict a collision if the 3-ellipsoid along the predicted trajectory intersects with a safety sphere around the quadrotor. We experimentally demonstrate that our method allows initiating evasive maneuvers early enough to avoid collisions. Abstract-We present a method to predict collisions with objects thrown at a quadrotor using a pair of dynamic vision sensors (DVS). Due to the micro-second temporal resolution of these sensors and the sparsity of their output, the object's trajectory can be estimated with minimal latency. Unlike standard cameras that send frames at a fixed frame rate, a DVS only transmits pixel-level brightness changes ("events") at the time they occur. Our method tracks spherical objects on the image plane using probabilistic trackers that are updated with each incoming event. The object's trajectory is estimated using an Extended Kalman Filter with a mixed state space that allows incorporation of both the object's dynamics and the measurement noise in the image plane. Using error-propagation techniques, we predict a collision if the 3σ-ellipsoid along the predicted trajectory intersects with a safety sphere around the quadrotor. We experimentally demonstrate that our method allows initiating evasive maneuvers early enough to avoid collisions.

EVO: A Geometric Approach to Event-Based 6-DOF Parallel Tracking and Mapping in Real Time

“…The times at which the room lights are switched off and on again are marked. As observed, EVO tracks the whole sequence with remarkable accuracy (6 cm drift in translation, and a few degrees (3 • ) in rotational drift, over a 30 m trajectory, that is, 0.2 % relative position error). Note that we do not perform any map or pose refinement (e.g., bundle adjustment); doing so would further reduce the drift.…”

“…(i) Notably, event cameras have negligible latency (microseconds), and so they have the potential to enable fast maneuvers of robotic platforms [2], [3], which are currently not possible with standard cameras because of the high latency of the sensing and processing pipeline (in the order of tens of milliseconds).…”

Visual Odometry for Pixel Processor Arrays

“…These devices produce an asynchronous stream of events encoding brightness changes incurred at specific pixels, allowing very fast response times and dynamic ranges. DVS have been demonstrated in agile UAVs maneuvers [17], orientation tracking [7], [11], visual odom- etry [13], 6DOF tracking and SLAM [12], [20]. However, DVS based systems still require the actual processing of visual data to be conducted on a separate external device, often via having to reconstruct a whole image.…”

Tracking control of a UAV with a parallel visual processor

“…The parallel nature of the SCAMP vision sensor allows various basic image processing tasks to be conducted with minimal computational overhead 1 . Efficient asynchronous flood fill of a binary black and white image is one such task, and it is this capability which is exploited in the target tracking algorithm implemented for this work.…”

“…Recent developments with dynamic vision sensors (DVS), offer some insight into the possibilities for developing more efficiently perceiving robotic systems. Examples for UAVs include works such as [1] for evasive maneuvers, [2] for agile visual odometry and [3] for landing from optic flow. When using a DVS, however, both basic and more visually complex tasks such as target tracking and/or combinations with lower visual competences require separate and more conventional processing architectures.…”