Multi-Bracket High Dynamic Range Imaging with Event Cameras

Nico, Messikommer,; Georgoulis, Stamatios; Gehrig, Daniel; Tulyakov, Stepan; Erbach, Julius; Bochicchio, Alfredo; Li, Yuanyou; Scaramuzza, Davide

doi:10.1109/cvprw56347.2022.00070

Cited by 27 publications

(12 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…More recently, deep-learning methods use both events and frames for image reconstruction are developed. They use intensity map reconstructed from events to guide frame-based HDR reconstruction [6], align bracket LDR images taken at different exposures [17] or interpolate between frames [7], [18]. These deep-learning methods require a large amount of training data and quality of the model depends heavily on the data source.…”

Section: Introductionmentioning

confidence: 99%

An Asynchronous Kalman Filter for Hybrid Event Cameras

Wang

Scheerlinck

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Event cameras are ideally suited to capture High Dynamic Range (HDR) visual information without blur but provide poor imaging capability for static or slowly varying scenes. Conversely, conventional image sensors measure absolute intensity of slowly changing scenes effectively but do poorly on HDR or quickly changing scenes. In this paper, we present an asynchronous linear filter architecture, fusing event and frame camera data, for HDR video reconstruction and spatial convolution that exploits the advantages of both sensor modalities. The key idea is the introduction of a state that directly encodes the integrated or convolved image information and that is updated asynchronously as each event or each frame arrives from the camera. The state can be read-off as-often-as and whenever required to feed into subsequent vision modules for real-time robotic systems. Our experimental results are evaluated on both publicly available datasets with challenging lighting conditions and fast motions, along with a new dataset with HDR reference that we provide. The proposed AKF pipeline outperforms other state-of-the-art methods in both absolute intensity error (69.4% reduction) and image similarity indexes (average 35.5% improvement). We also demonstrate the integration of image convolution with linear spatial kernels Gaussian, Sobel, and Laplacian as an application of our architecture.

show abstract

Section: Introductionmentioning

confidence: 99%

An Asynchronous Kalman Filter for Hybrid Event Cameras

Wang

Scheerlinck

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

show abstract

“…The HDR of events makes it naturally more advantageous to reconstruct an HDR image/video. The predominant methods can be divided into two main categories: event-based HDR image/video HDR methods [89], [120], [121] and eventguided image/video HDR methods (a hybrid of event and frame data) [122], [123], [124]. Event-based image/video HDR typically employs the idea of event-to-image translation-reconstructing HDR images from events, as mentioned in Sec.…”

Section: Event-based Deep Image/video Hdr Insightmentioning

confidence: 99%

“…Event-guided image/video also follows these two paradigms. [122], [123], [124] explored the potential of merging both events and frames for this task, as shown in Fig. 11(c).…”

Section: Event-based Deep Image/video Hdr Insightmentioning

confidence: 99%

“…This framework addresses the gaps in spatial resolution, dynamic range, and color representation of the hybrid sensor system to pursue a better fusion. By contrast, EHDR [124] is the first multi-exposure HDR imaging framework that combines bracketed LDR images and synchronized events to recover an HDR image. To alleviate the impact of scene motion between exposures, EHDR employs events to learn a deformable convolution kernel, which can align feature maps from images with different exposure time.…”

Section: Event-based Deep Image/video Hdr Insightmentioning

confidence: 99%

“…Representative DL-based HDR imaging methods. (a) RNNbased methods[89],[120] and (b) GAN-based method[121] that use events only, and (c) event-frame fusion[122],[123],[124]. (latent embedding denotes information learned by feature extractor, filter,etc.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Pan‐risk factor for a comprehensive cardiovascular health management

Zheng

et al. 2022

Journal of Diabetes

View full text Add to dashboard Cite

Cardiovascular diseases (CVDs) have become the leading cause of death in China. CVDs are mainly caused by multiple well‐known modifiable risk factors that are affected by socioeconomic and environmental determinants, lifestyle and behavioral choices, and familial and genetic predispositions. With more risk factors proved to be associated with CVD occurrence, the concept “pan‐risk factor” is proposed in this review to indicate all discovered and yet‐to‐be‐discovered CVD risk factors for comprehensive primary prevention of CVD. Recognizing more factors and their roles in CVD development and progression is the first step in reducing the ever‐increasing burden of CVD. This review is an overview of the pan‐risk factor whose associations with CVD outcomes have been established. Along with the accumulation of scientific evidence, an increasing number of risk factors will be discovered and included in the list of pan‐risk factors.

show abstract