Event cameras capture changes of illumination in the observed scene rather than accumulating light to create images. Thus, they allow for applications under high-speed motion and complex lighting conditions, where traditional framebased sensors show their limits with blur and over-or underexposed pixels. Thanks to these unique properties, they represent nowadays an highly attractive sensor for ITS-related applications. Event-based optical flow (EBOF) has been studied following the rise in popularity of these neuromorphic cameras. The recent arrival of high-definition neuromorphic sensors, however, challenges the existing approaches, because of the increased resolution of the events pixel array and a much higher throughput. As an answer to these points, we propose an optimized framework for computing optical flow in real-time with both low-and high-resolution event cameras. We formulate a novel dense representation for the sparse events flow, in the form of the "inverse exponential distance surface". It serves as an interim frame, designed for the use of proven, state-of-the-art frame-based optical flow computation methods. We evaluate our approach on both low-and high-resolution driving sequences, and show that it often achieves better results than the current state of the art, while also reaching higher frame rates, 250Hz at 346×260 pixels and 77Hz at 1280×720 pixels.Index Terms-Machine vision, neuromorphic cameras, optical flow, real-time applications.• a specific pipeline-based architecture, for computing realtime optical flow using the events from low-or highresolution neuromorphic sensors; • the formulation of a novel dense "inverse exponential distance surface", that acts as the frame-based representation computed from the events, able to feed any image-based optical flow method; • a coherent choice of algorithms and methods together for all the steps up to the fast frame-based state-of-theart optical flow (with temporal smoothing to fit well with potentially noisy input events); • we finally build and share a complementary highdefinition event-based dataset of indoor sequences with high-speed movements, used as part of our evaluation. Videos accompanying this article, showing results for both low-and high-resolution data, are available at https://youtube.com/playlist?list= PLLL0eWAd6OXBRXli-tB1NREdhBElAxisD.

No abstract

Event cameras do not produce images, but rather a continuous flow of events, which encode changes of illumination for each pixel independently and asynchronously. While they output temporally rich information, they lack any depth information which could facilitate their use with other sensors. LiDARs can provide this depth information, but are by nature very sparse, which makes the depth-to-event association more complex. Furthermore, as events represent changes of illumination, they might also represent changes of depth; associating them with a single depth is therefore inadequate. In this work, we propose to address these issues by fusing information from an event camera and a LiDAR using a learning-based approach to estimate accurate dense depth maps. To solve the "potential change of depth" problem, we propose here to estimate two depth maps at each step: one "before" the events happen, and one "after" the events happen. We further propose to use this pair of depths to compute a depth difference for each event, to give them more context. We train and evaluate our network, ALED, on both synthetic and real driving sequences, and show that it is able to predict dense depths with an error reduction of up to 61% compared to the current state of the art. We also demonstrate the quality of our 2-depths-toevent association, and the usefulness of the depth difference information. Finally, we release SLED, a novel synthetic dataset comprising events, LiDAR point clouds, RGB images, and dense depth maps.

Event cameras capture changes of illumination in the observed scene rather than accumulating light to create images. Thus, they allow for applications under high-speed motion and complex lighting conditions, where traditional framebased sensors show their limits with blur and over-or underexposed pixels. Thanks to these unique properties, they represent nowadays an highly attractive sensor for ITS-related applications. Event-based optical flow (EBOF) has been studied following the rise in popularity of these neuromorphic cameras. The recent arrival of high-definition neuromorphic sensors, however, challenges the existing approaches, because of the increased resolution of the events pixel array and a much higher throughput. As an answer to these points, we propose an optimized framework for computing optical flow in real-time with both low-and high-resolution event cameras. We formulate a novel dense representation for the sparse events flow, in the form of the "inverse exponential distance surface". It serves as an interim frame, designed for the use of proven, state-of-the-art frame-based optical flow computation methods. We evaluate our approach on both low-and high-resolution driving sequences, and show that it often achieves better results than the current state of the art, while also reaching higher frame rates, 250Hz at 346×260 pixels and 77Hz at 1280×720 pixels.Index Terms-Machine vision, neuromorphic cameras, optical flow, real-time applications.• a specific pipeline-based architecture, for computing realtime optical flow using the events from low-or highresolution neuromorphic sensors; • the formulation of a novel dense "inverse exponential distance surface", that acts as the frame-based representation computed from the events, able to feed any image-based optical flow method; • a coherent choice of algorithms and methods together for all the steps up to the fast frame-based state-of-theart optical flow (with temporal smoothing to fit well with potentially noisy input events); • we finally build and share a complementary highdefinition event-based dataset of indoor sequences with high-speed movements, used as part of our evaluation.Videos accompanying this article, showing results for both low-and high-resolution data, are available at https://youtube.com/playlist?list= PLLL0eWAd6OXBRXli-tB1NREdhBElAxisD.1 Considering a car at 120Km/h, if we tolerate to drive a distance of 1 meter to achieve perception analysis, it means a maximum latency of 30ms.