EVReflex: Dense Time-to-Impact Prediction for Event-based Obstacle Avoidance

Walters, Celyn; Hadfield, Simon

doi:10.1109/iros51168.2021.9636327

Cited by 11 publications

(11 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Energy Efficiency [160] Leveraged neuromorphic hardware for energy-efficient and robust obstacle avoidance. Sensor Fusion [161], [162] Combined event cameras with lidar for innovative Time-To-Impact estimation, improving accuracy in dynamic environments. High-Speed Reactive Control [163] Pioneered fast obstacle avoidance suitable for high-speed sensor-based reactive control.…”

Section: Challenge Addressedmentioning

confidence: 99%

Event Cameras in Automotive Sensing: A Review

Shariff,

Dilmaghani,

Kielty

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Event cameras (EC) represent a paradigm shift and are emerging as valuable tools in the automotive industry, particularly for in-cabin and out-of-cabin monitoring. These cameras capture pixel intensity changes as "events" with ultra-low latency, making them suitable for real-time applications. In the context of in-cabin monitoring, EC offer solution for driver and passenger tracking, enhancing safety and comfort. For out-of-cabin monitoring, they excel in tracking objects and detecting potential hazards on the road. This article explores the applications, benefits, and challenges of event cameras in these two critical domains within the automotive industry. This review also highlights relevant datasets and methodologies, enabling researchers to make informed decisions tailored to their specific vehicular-technology and place their work in the broader context of EC sensing. Through an exploration of the hardware, the complexities of data processing, and customized algorithms for both in-cabin and out-of-cabin surveillance, this paper outlines a framework encompassing methodologies, tools, and datasets critical for the implementation of event camera sensing in automotive systems. INDEX TERMSNeuromorphic Event Camera, Event based Vision, Driver Monitoring System, Out-of-cabin Monitoring, In-Cabin Monitoring.

show abstract

Section: Challenge Addressedmentioning

confidence: 99%

Event Cameras in Automotive Sensing: A Review

Shariff,

Dilmaghani,

Kielty

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Attempts have also been made to directly learn the structure of the scene and the movement of the camera from the event cameras [18]. Another line of research learns dense time-to-collision (TTC) from monocular event sequences [19]. In addition to these perception-focused works, end-to-end learning of control input from event data has enabled complicated control tasks such as UAV navigation [20].…”

Section: B Event Camerasmentioning

confidence: 99%

EV-Catcher: High-Speed Object Catching Using Low-Latency Event-Based Neural Networks

Wang

Cladera

Bisulco

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Event-based sensors have recently drawn increasing interest in robotic perception due to their lower latency, higher dynamic range, and lower bandwidth requirements compared to standard CMOS-based imagers. These properties make them ideal tools for real-time perception tasks in highly dynamic environments. In this work, we demonstrate an application where event cameras excel: accurately estimating the impact location of fast-moving objects. We introduce a lightweight event representation called Binary Event History Image (BEHI) to encode event data at low latency, as well as a learning-based approach that allows real-time inference of a confidence-enabled control signal to the robot. To validate our approach, we present an experimental catching system in which we catch fast-flying pingpong balls. We show that the system is capable of achieving a success rate of 81% in catching balls targeted at different locations, with a velocity of up to 13 m/s even on computeconstrained embedded platforms such as the Nvidia Jetson NX.

show abstract

“…[31] showed how to land a spacecraft using event cameras by computing τ from the divergence of optical flow. [32] fuses information from a depth camera and τ to compute "time-to-impact" which can in-turn be used to dodge dynamic obstacles without prior knowledge of scene geometry or obstacles. In robotics, most methods that perform optical flow based control use initial height estimates either implicitly or explicitly, as remarked in [15].…”

Section: Time-to-contactmentioning

confidence: 99%

Fast Active Monocular Distance Estimation from Time-to-Contact

Levi¹,

Sanket²,

Fermüller³

et al. 2022

Preprint

View full text Add to dashboard Cite

Distance estimation is fundamental for a variety of robotic applications including navigation, manipulation and planning. Inspired by the mammal's visual system, which gazes at specific objects (active fixation), and estimates when the object will reach it (time-to-contact), we develop a novel constraint between time-to-contact, acceleration, and distance that we call the τ -constraint. It allows an active monocular camera to estimate depth using timeto-contact and inertial measurements (linear accelerations and angular velocities) within a window of time.Our work differs from other approaches by focusing on patches instead of feature points. This is, because the change in the patch area determines the time-to-contact directly. The result enables efficient estimation of distance while using only a small portion of the image, leading to a large speedup.We successfully validate the proposed τ -constraint in the application of estimating camera position with a monocular grayscale camera and an Inertial Measurement Unit (IMU). Specifically, we test our method on different realworld planar objects over trajectories 8-40 seconds in duration and 7-35 meters long. Our method achieves 8.5 cm Average Trajectory Error (ATE) while the popular Visual-Inertial Odometry methods VINS-Mono and ROVIO achieve 12.2 and 16.9 cm ATE respectively. Additionally, our implementation runs 27× faster than VINS-Mono's and 6.8× faster than ROVIO's. We believe these results indicate the τ -constraints potential to be the basis of robust, sophisticated algorithms for a multitude of applications involving an active camera and an IMU.

show abstract

EVReflex: Dense Time-to-Impact Prediction for Event-based Obstacle Avoidance

Cited by 11 publications

References 19 publications

Event Cameras in Automotive Sensing: A Review

Event Cameras in Automotive Sensing: A Review

EV-Catcher: High-Speed Object Catching Using Low-Latency Event-Based Neural Networks

Fast Active Monocular Distance Estimation from Time-to-Contact

Contact Info

Product

Resources

About