Data Association Between Event Streams and Intensity Frames Under Diverse Baselines

Zhang, Dehao; Ding, Qiankun; Duan, Peiqi; Zhou, Chu; Shi, Boxin

doi:10.1007/978-3-031-20071-7_5

Cited by 4 publications

(1 citation statement)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Blurry Effects. When perceiving the same light field, events and intensities exhibit in different modalities but implicitly share common structures, enabling pixel-level correspondence [15]. However, such correspondence becomes weak and ambiguous when intensity images appear blurry.…”

Section: Introductionmentioning

confidence: 99%

Learning Event-Based Motion Deblurring

Jiang

Zhang

Zou³

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

149

View full text Add to dashboard Cite

Due to the extremely low latency, events have been recently exploited to supplement lost information for motion deblurring. Existing approaches largely rely on the perfect pixelwise alignment between intensity images and events, which is not always fulfilled in the real world. To tackle this problem, we propose a novel coarse-to-fine framework, named NETwork of Event-based motion Deblurring with STereo event and intensity cameras (St-EDNet), to recover high-quality images directly from the misaligned inputs, consisting of a single blurry image and the concurrent event streams. Specifically, the coarse spatial alignment of the blurry image and the event streams is first implemented with a cross-modal stereo matching module without the need for ground-truth depths. Then, a dual-feature embedding architecture is proposed to gradually build the fine bidirectional association of the coarsely aligned data and reconstruct the sequence of the latent sharp images. Furthermore, we build a new dataset with STereo Event and Intensity Cameras (StEIC), containing real-world events, intensity images, and dense disparity maps. Experiments on real-world datasets demonstrate the superiority of the proposed network over state-of-the-art methods. The code and dataset are available at https://mingyuan-lin.github.io/St-ED web/.

show abstract

Section: Introductionmentioning

confidence: 99%