Tracking from a moving platform with the Dynamic Vision Sensor

Abstract

The Dynamic Vision Sensor (DVS) is an imaging sensor that processes the incident irradiance image and outputs temporal log irradiance changes in the image, such as those generated by moving target(s) and/or the moving sensor platform. From a static platform, this enables the DVS to cancel out background clutter and greatly decrease the sensor bandwidth required to track temporal changes in a scene. However, the sensor bandwidth advantage is lost when imaging a scene from a moving platform due to platform motion causing optical flow in the background. Imaging from a moving platform has been utilized in many recently reported applications of this sensor. However, this approach inherently outputs background clutter generated from optical flow, and as such this approach has limited spatio-temporal resolution and is of limited utility for target tracking applications. In this work we present a new approach to moving target tracking applications with the DVS. Essentially, we propose modifying the incident image to cancel out optical flow due to platform motion, thereby removing background clutter and recovering the bandwidth performance advantage of the DVS. We propose that such improved performance can be accomplished by integrating a hardware tracking and stabilization subsystem with the DVS. Representative simulation scenarios are used to quantify the performance of the proposed approach to clutter cancellation and improved sensor bandwidth.

Hybrid event and frame-based system for target detection, tracking, and identification

“…2 They also operate at a read-out bandwidth that is typically an order of magnitude less than an equivalently sized frame-based sensor. 3 The EBS technology has only become commercially available recently, has already seen use in several applications, such as robotics, [4][5][6] autonomous driving, 7 object tracking, [8][9][10] and high speed video. [11][12][13] Although the EBS is inherently well-suited for detecting moving objects with minimal strain on computational resources, the noise and lack of intensity information results in potentially lower task performance than a frame-based sensor for some tasks.…”

Optical contrast amplification for enhancing event-based vision

“…In contrast to frame-based imagers, EBS operates without a synchronous frame capture process. With the EBS, this different behavior leads to an overall reduction in bandwidth that can lead to an increase in spatio-temporal resolution 10 . Given that logged events are always relative (log) irradiance changes, dynamic ranges of up to 120 dB 11 can be achieved EBS sensors as opposed to typical conventional imagers with a range up to 60 dB 9 .…”