EventZoom: Learning to Denoise and Super Resolve Neuromorphic Events

Duan, Peiqi; Wang, Zihao W.; Zhou, Xinyu; Ma, Yi; Shi, Boxin

doi:10.1109/cvpr46437.2021.01263

Cited by 53 publications

(22 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Wang et al [11] converted spikes into intensity images and then used a 3-phase GANs for super-resolution reconstruction. In these related work, event up-sampling can improve the performance of downstream tasks, such as object tracking [9] and reconstruction [11], which proves the event up-sampling is meaningful.…”

Section: Related Workmentioning

confidence: 85%

“…The high temporal resolution of the event camera limits the improvement of its spatial resolution, resulting in the low spatial resolution of the existing event cameras. There are some works [8,9] to improve the spatial resolution of events, aiming to generate high spatial resolution events from low spatial resolution events. The general process is to first convert asynchronous events into images, then obtain high-resolution images through imagebased super-resolution algorithms, and finally randomly generate high-resolution events based on the high-resolution images.…”

Section: Related Workmentioning

confidence: 99%

“…Then, we up-sample new events on the correct motion trajectory. Some existing work [8,9] randomly generate events through Poisson Process. But Poisson Process is memoryless and cannot guarantee the consistency of newly generated events with the original events.…”

Section: Algorithm 1: Event Temporal Up-samplingmentioning

confidence: 99%

See 2 more Smart Citations

Temporal Up-Sampling for Asynchronous Events

Xiang

Zhu

et al. 2022

2022 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

The event camera is a novel bio-inspired vision sensor. When the brightness change exceeds the preset threshold, the sensor generates events asynchronously. The number of valid events directly affects the performance of event-based tasks, such as reconstruction, detection, and recognition. However, when in low-brightness or slow-moving scenes, events are often sparse and accompanied by noise, which poses challenges for event-based tasks. To solve these challenges, we propose an event temporal up-sampling algorithm 1 to generate more effective and reliable events. The main idea of our algorithm is to generate up-sampling events on the event motion trajectory. First, we estimate the event motion trajectory by contrast maximization algorithm and then up-sampling the events by temporal point processes. Experimental results show that upsampling events can provide more effective information and improve the performance of downstream tasks, such as improving the quality of reconstructed images and increasing the accuracy of object detection.

show abstract

Section: Related Workmentioning

confidence: 85%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Temporal Up-Sampling for Asynchronous Events

Xiang

Zhu

et al. 2022

2022 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

show abstract

“…object tracking [26], image reconstruction [26], and segmentation [27]. DVS produces noise due to various reasons.…”

Section: A Event Denoisingmentioning

confidence: 99%

“…Several event noise reduction methods have been proposed in the literature. These methods can be categorized into conventional methods [21], [22], [23], [28], [29], [30] and deep learning methods [20], [26], [31]. The most widely prevalent filtering approach is based on the nearest neighbor (NNb) method and hence on spatiotemporal correlation [22], [23], [28].…”

Section: A Event Denoisingmentioning

confidence: 99%

Neuromorphic Camera Denoising Using Graph Neural Network-Driven Transformers

Alkendi

Azzam

Ayyad

et al. 2024

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Neuromorphic vision is a bio-inspired technology 1 that has triggered a paradigm shift in the computer vision 2 community and is serving as a key enabler for a wide range of 3 applications. This technology has offered significant advantages, 4 including reduced power consumption, reduced processing needs, 5 and communication speedups. However, neuromorphic cameras 6 suffer from significant amounts of measurement noise. This 7 noise deteriorates the performance of neuromorphic event-based 8 perception and navigation algorithms. In this article, we propose 9 a novel noise filtration algorithm to eliminate events that do 10 not represent real log-intensity variations in the observed scene. 11 We employ a graph neural network (GNN)-driven transformer 12 algorithm, called GNN-Transformer, to classify every active event 13 pixel in the raw stream into real log-intensity variation or 14 noise. Within the GNN, a message-passing framework, referred 15 to as EventConv, is carried out to reflect the spatiotemporal 16 correlation among the events while preserving their asynchronous 17 nature. We also introduce the known-object ground-truth label-18 ing (KoGTL) approach for generating approximate ground-truth 19 labels of event streams under various illumination conditions. 20 KoGTL is used to generate labeled datasets, from experiments 21 recorded in challenging lighting conditions, including moon light. 22 These datasets are used to train and extensively test our proposed 23 algorithm. When tested on unseen datasets, the proposed algo-24 rithm outperforms state-of-the-art methods by at least 8.8% in 25 terms of filtration accuracy. Additional tests are also conducted 26 on publicly available datasets (ETH Zürich Color-DAVIS346 27 datasets) to demonstrate the generalization capabilities of the 28 proposed algorithm in the presence of illumination variations 29 and different motion dynamics. Compared to state-of-the-art 30 solutions, qualitative results verified the superior capability of 31 the proposed algorithm to eliminate noise while preserving 32 meaningful events in the scene.

show abstract