Learning to Reconstruct High Speed and High Dynamic Range Videos from Events

Zou, Yunhao; Zheng, Yinqiang; Takatani, Tsuyoshi; Fu, Ying

doi:10.1109/cvpr46437.2021.00206

Cited by 56 publications

(20 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The above event-to-image deep learning methods are to a large extent black boxes, with a lot of effort spent not only on loss design and architecture search but also on dataset preparation [20], [29], [33] to train the ANNs and make them learn the desired transformation. Different from these methods, our approach does not train any event-to-image ANN.…”

Section: Related Workmentioning

confidence: 99%

Formulating Event-based Image Reconstruction as a Linear Inverse Problem with Deep Regularization using Optical Flow

Zhang

Yezzi

Gallego

2022

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Event cameras are novel bio-inspired sensors that measure per-pixel brightness differences asynchronously. Recovering brightness from events is appealing since the reconstructed images inherit the high dynamic range (HDR) and high-speed properties of events; hence they can be used in many robotic vision applications and to generate slow-motion HDR videos. However, state-of-the-art methods tackle this problem by training an event-to-image Recurrent Neural Network (RNN), which lacks explainability and is difficult to tune. In this work we show, for the first time, how tackling the combined problem of motion and brightness estimation leads us to formulate event-based image reconstruction as a linear inverse problem that can be solved without training an image reconstruction RNN. Instead, classical and learning-based regularizers are used to solve the problem and remove artifacts from the reconstructed images. The experiments show that the proposed approach generates images with visual quality on par with state-of-the-art methods despite only using data from a short time interval. State-of-the-art results are achieved using an image denoising Convolutional Neural Network (CNN) as the regularization function. The proposed regularized formulation and solvers have a unifying character because they can be applied also to reconstruct brightness from the second derivative. Additionally, the formulation is attractive because it can be naturally combined with super-resolution, motion-segmentation and color demosaicing. Code is available at https://github.com/tub-rip/event based image rec inverse problem

show abstract

Section: Related Workmentioning

confidence: 99%

Formulating Event-based Image Reconstruction as a Linear Inverse Problem with Deep Regularization using Optical Flow

Zhang

Yezzi

Gallego

2022

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Event-based methods [41,53,65,69] reconstruct intensity frames from events using deep learning. These methods do not explicitly target HDR imaging, but they rather reconstruct HDR-like images as a consequence of using events.…”

Section: Related Workmentioning

confidence: 99%

“…The resulting data have high temporal resolution, HDR and do not suffer from motion blur. Recent works have leveraged the outstanding properties of event cameras to generate high-speed video reconstructions with HDR properties from events [41,53,65,69]. Nonetheless, event cameras only measure changes in brightness, and thus global image reconstruction from events is ill-posed.…”

Section: Introductionmentioning

confidence: 99%

Multi-Bracket High Dynamic Range Imaging with Event Cameras

Nico¹,

Georgoulis²,

Gehrig³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To solve the third challenge, GANs, the double integral model [96], and RNNs are applied to avoid generating blurred results and obtain high-speed and HDR videos from events in [14], [91], [97], [98], [99]. As for generating super-resolution (SR) images/videos from events, we divide the prevailing works into three categories, including optimization-based [60], supervised [61], [62], [100], [101], and adversarial learning methods [102], [103].…”

Section: Eventsmentioning

confidence: 99%

Pan‐risk factor for a comprehensive cardiovascular health management

Zheng

et al. 2022

Journal of Diabetes

View full text Add to dashboard Cite

Cardiovascular diseases (CVDs) have become the leading cause of death in China. CVDs are mainly caused by multiple well‐known modifiable risk factors that are affected by socioeconomic and environmental determinants, lifestyle and behavioral choices, and familial and genetic predispositions. With more risk factors proved to be associated with CVD occurrence, the concept “pan‐risk factor” is proposed in this review to indicate all discovered and yet‐to‐be‐discovered CVD risk factors for comprehensive primary prevention of CVD. Recognizing more factors and their roles in CVD development and progression is the first step in reducing the ever‐increasing burden of CVD. This review is an overview of the pan‐risk factor whose associations with CVD outcomes have been established. Along with the accumulation of scientific evidence, an increasing number of risk factors will be discovered and included in the list of pan‐risk factors.

show abstract

Learning to Reconstruct High Speed and High Dynamic Range Videos from Events

Cited by 56 publications

References 25 publications

Formulating Event-based Image Reconstruction as a Linear Inverse Problem with Deep Regularization using Optical Flow

Formulating Event-based Image Reconstruction as a Linear Inverse Problem with Deep Regularization using Optical Flow

Multi-Bracket High Dynamic Range Imaging with Event Cameras

Pan‐risk factor for a comprehensive cardiovascular health management

Contact Info

Product

Resources

About