Continuous-Time Intensity Estimation Using Event Cameras

Scheerlinck, Cedric; Barnes, Nick; Mahony, Robert

doi:10.1007/978-3-030-20873-8_20

Cited by 133 publications

(219 citation statements)

References 34 publications

Supporting

Mentioning

218

Contrasting

Unclassified

Order By: Relevance

“…Figs. 21, 22 and 23 present HDR reconstruction results on various publicly available datasets [5], [43], [45]. Further results are shown in the supplementary video which conveys these results in a better form than still images.…”

Section: D1 Results On Synthetic Event Datamentioning

confidence: 94%

“…19. Qualitative comparison of our reconstruction method with two recent competing approaches, MR [4] and HF [5], on sequences from [38], which contain ground truth frames from a DAVIS240C sensor. Our method successfully reconstructs fine details (textures in the second and third row) compared to other methods, while avoiding ghosting effects (particulary visible in the shapes sequences on the fourth row).…”

Section: D1 Results On Synthetic Event Datamentioning

confidence: 99%

“…Nonetheless, our reconstruction does not suffer from glare, and reveals the circular shape of the sun, which is lost in the frame. The video sequences, along with additional results on sequences from previously released datasets [5], [43], are available in the supplementary material.…”

Section: Water Balloonmentioning

confidence: 99%

“…Alternatively, our network could also be used to generate high-quality images at low framerates (i.e. using large batches of events), which could be fused with event data using a complementary filter [5] to synthesize high-quality videos at very high framerate more efficiently.…”

Section: Computational Efficiencymentioning

confidence: 99%

“…Naive integration of the event stream leads to very fast degradation of image quality due to accumulating noise. As a remedy, earlier works have proposed hand-crafted image priors to constrain the problem [2], [3], [4], arXiv:1906.07165v1 [cs.CV] 15 Jun 2019 [5]. However, these priors make strong assumptions about the statistics of natural images, leading to unrealistic reconstructions and artifacts.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

High Speed and High Dynamic Range Video with an Event Camera

Rebecq

Ranftl

Koltun

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

389

339

View full text Add to dashboard Cite

Event cameras are novel sensors that report brightness changes in the form of a stream of asynchronous "events" instead of intensity frames. They offer significant advantages with respect to conventional cameras: high temporal resolution, high dynamic range, and no motion blur. While the stream of events encodes in principle the complete visual signal, the reconstruction of an intensity image from a stream of events is an ill-posed problem in practice. Existing reconstruction approaches are based on hand-crafted priors and strong assumptions about the imaging process as well as the statistics of natural images. In this work we propose to learn to reconstruct intensity images from event streams directly from data instead of relying on any hand-crafted priors. We propose a novel recurrent network to reconstruct videos from a stream of events, and train it on a large amount of simulated event data. During training we propose to use a perceptual loss to encourage reconstructions to follow natural image statistics. We further extend our approach to synthesize color images from color event streams. Our quantitative experiments show that our network surpasses state-of-the-art reconstruction methods by a large margin in terms of image quality (> 20%), while comfortably running in real-time. We show that the network is able to synthesize high framerate videos (> 5,000 frames per second) of high-speed phenomena (e.g. a bullet hitting an object) and is able to provide high dynamic range reconstructions in challenging lighting conditions. As an additional contribution, we demonstrate the effectiveness of our reconstructions as an intermediate representation for event data. We show that off-the-shelf computer vision algorithms can be applied to our reconstructions for tasks such as object classification and visual-inertial odometry and that this strategy consistently outperforms algorithms that were specifically designed for event data. We release the reconstruction code and a pre-trained model to enable further research.

show abstract

Section: D1 Results On Synthetic Event Datamentioning

confidence: 94%

Section: D1 Results On Synthetic Event Datamentioning

confidence: 99%

Section: Water Balloonmentioning

confidence: 99%

Section: Computational Efficiencymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

High Speed and High Dynamic Range Video with an Event Camera

Rebecq

Ranftl

Koltun

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

389

339

View full text Add to dashboard Cite

show abstract

T‐ESVO: Improved Event‐Based Stereo Visual Odometry via Adaptive Time‐Surface and Truncated Signed Distance Function

Liu

Shi

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

The emerging event cameras have the potential to be an excellent complement for standard cameras within various visual tasks, especially in illumination‐changing environments or situations requiring high‐temporal resolution. Herein, an event‐based stereo visual odometry (VO) system via adaptive time‐surface (TS) and truncated signed distance function (TSDF), namely, T‐ESVO, is proposed . The system consists of three carefully designed components, including the event processing unit, the mapping unit, and the tracking unit. Specifically, the event processing unit adopts a novel spatial–temporal adaptive TS that can deal with different camera motions in various environments. The mapping unit introduces the TSDF to describe the 3D representation of environments and achieves depth estimation based on the global historical depth information contained in the environmental TSDF description. The tracking unit achieves the 6‐DoF pose estimation through an 3D–2D registration method based on the left/right TS selection mechanism and the depth point selection mechanism. The effectiveness and robustness of the proposed system are evaluated on various datasets, and the experimental results show that T‐ESVO achieves good performance in both accuracy and robustness when compared with other state‐of‐the‐art event‐based stereo VO systems.

show abstract

Video Synthesis from Intensity and Event Frames

Pini

Borghi

Vezzani

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Event cameras, neuromorphic devices that naturally respond to brightness changes, have multiple advantages with respect to traditional cameras. However, the difficulty of applying traditional computer vision algorithms on event data limits their usability. Therefore, in this paper we investigate the use of a deep learning-based architecture that combines an initial grayscale frame and a series of event data to estimate the following intensity frames. In particular, a fully-convolutional encoder-decoder network is employed and evaluated for the frame synthesis task on an automotive event-based dataset. Performance obtained with pixel-wise metrics confirms the quality of the images synthesized by the proposed architecture.

show abstract

Continuous-Time Intensity Estimation Using Event Cameras

Cited by 133 publications

References 34 publications

High Speed and High Dynamic Range Video with an Event Camera

High Speed and High Dynamic Range Video with an Event Camera

T‐ESVO: Improved Event‐Based Stereo Visual Odometry via Adaptive Time‐Surface and Truncated Signed Distance Function

Video Synthesis from Intensity and Event Frames

Contact Info

Product

Resources

About