Event-Based Vision: A Survey

Abstract

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of µs), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world.

A survey on multi-robot coverage path planning for model reconstruction and mapping

“…These kind of sensors can operate in different light intensities, and provide the property of low latency, and power consumption. An extensive review of all the algorithms, applications, and datasets is provided in [27]. Moreover, using heterogeneous platforms facilitates dividing complex tasks in harsh environments and enhance flexibility and mobility.…”

Event-Driven Visual-Tactile Sensing and Learning for Robots

“…Rather, our sensor and learning method are both event-driven. 2 Available at https://clear-nus.github.io/visuotactile/ B. Event-based Perception: Sensors and Learning Work on event-based perception has focused primarily on vision (see [29] for a comprehensive survey). This emphasis on vision can be attributed both to its applicability across many tasks, as well as the recent availability of event cameras such as the DVS and Prophesee Onboard.…”

“…Event-based sensors have been successfully used in conjunction with deep learning techniques [29]. The binary events are first converted into real-valued tensors, which are processed downstream by deep ANNs.…”

Feature-based visual simultaneous localization and mapping: a survey

“…Due to its unique way of acquiring information from the environment, a paradigm shift is necessary to construct algorithms that accommodate such information. Eventbased SLAM is beyond the scope of this review paper and interested readers are referred to the comprehensive survey in [41].…”