Event Enhanced High-Quality Image Recovery

Wang, Bishan; He, Junguo; Yu, Lei; Xia, Gui-Song; Yang, Wen

doi:10.1007/978-3-030-58601-0_10

Cited by 84 publications

(54 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…An event occurs whenever a change in log-scale intensity exceeds a threshold. A natural choice is to encode events in a spatial-temporal 3D volume to a voxel grid [56,86,87] or event frame [20,55] or multi-channel images [37,65,68]. In this paper, we represent events to multi-channel event images as the inputs to the DNNs.…”

Section: The Proposed Method: Evdistillmentioning

confidence: 99%

“…Meanwhile, another line of research focuses on the low-level prediction tasks, such as optical flow estimation [17,20,61,87], stereo depth estimation [64,87] on MVSEC dataset [85]. In addition, [44,50,56,58,61,68] attempted to reconstruct intensity image/video from events using camera simulator [45,54], and [43,65,67] tried to reconstruct high-resolution images. In contrast to image/video reconstruction from events, [19] proposed to generate events from video frames.…”

Section: Related Workmentioning

confidence: 99%

“…While [85,87] have explored unsupervised learning, they only focus on the pixellevel prediction tasks, e.g., depth estimation. Another line of research has reconstructed intensity images from events [43,56,61,65,67,68], and these images have been used to learn DNNs on end-tasks, e.g., object recognition [56]; however, annotated labels are still needed, and extra latency is introduced in the inference time.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation

Wang¹,

Chae²,

Yoon³

et al. 2021

Preprint

View full text Add to dashboard Cite

Event cameras sense per-pixel intensity changes and produce asynchronous event streams with high dynamic range and less motion blur, showing advantages over the conventional cameras. A hurdle of training event-based models is the lack of large qualitative labeled data. Prior works learning end-tasks mostly rely on labeled or pseudolabeled datasets obtained from the active pixel sensor (APS) frames; however, such datasets' quality is far from rivaling those based on the canonical images. In this paper, we propose a novel approach, called EvDistill, to learn a student network on the unlabeled and unpaired event data (target modality) via knowledge distillation (KD) from a teacher network trained with large-scale, labeled image data (source modality). To enable KD across the unpaired modalities, we first propose a bidirectional modality reconstruction (BMR) module to bridge both modalities and simultaneously exploit them to distill knowledge via the crafted pairs, causing no extra computation in the inference. The BMR is improved by the end-tasks and KD losses in an end-to-end manner. Second, we leverage the structural similarities of both modalities and adapt the knowledge by matching their distributions. Moreover, as most prior feature KD methods are uni-modality and less applicable to our problem, we propose to leverage an affinity graph KD loss to boost the distillation. Our extensive experiments on semantic segmentation and object recognition demonstrate that EvDistill achieves significantly better results than the prior works and KD with only events and APS frames.

show abstract

Section: The Proposed Method: Evdistillmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation

Wang¹,

Chae²,

Yoon³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, some challenges remain. First, reconstructing HDR images using DNNs induces considerable inference latency [123], [138]. Second, as events predominantly reflect the edge information of scenes and suffer from noise due to abrupt intensity changes in extreme lighting conditions, the reconstructed HDR images are degraded by noise and loss of textural details.…”

Section: Reconstruction-based Hdr Imagingmentioning

confidence: 99%

Deep Learning for HDR Imaging: State-of-the-Art and Future Trends

Wang

Yoon

2021

Preprint

View full text Add to dashboard Cite

High dynamic range (HDR) imaging is a technique that allows an extensive dynamic range of exposures, which is important in image processing, computer graphics, and computer vision. In recent years, there has been a significant advancement in HDR imaging using deep learning (DL). This study conducts a comprehensive and insightful survey and analysis of recent developments in deep HDR imaging methodologies. We hierarchically and structurally group existing deep HDR imaging methods into five categories based on (1) number/domain of input exposures, (2) number of learning tasks, (3) novel sensor data, (4) novel learning strategies, and (5) applications. Importantly, we provide a constructive discussion on each category regarding its potential and challenges. Moreover, we review some crucial aspects of deep HDR imaging, such as datasets and evaluation metrics. Finally, we highlight some open problems and point out future research directions.

show abstract

“…The generation of landscape design renderings is greatly affected by the color matching, design layout and other information in the original image [12][13][14]. Based on high quality garden landscape images (GLIs), designers can effectively visualize the design intent and conception of the actual landscape effect [16][17][18][19]. In the real world, GLIs taken in an unfavorable shooting environment are insufficiently exposed, unevenly illuminated, and generally dark.…”

Section: Introductionmentioning

confidence: 99%

Color Enhancement of Low Illumination Garden Landscape Images

Zhang¹,

Lü²,

Liu³

et al. 2021

View full text Add to dashboard Cite

The unfavorable shooting environment severely hinders the acquisition of actual landscape information in garden landscape design. Low quality, low illumination garden landscape images (GLIs) can be enhanced through advanced digital image processing. However, the current color enhancement models have poor applicability. When the environment changes, these models are easy to lose image details, and perform with a low robustness. Therefore, this paper tries to enhance the color of low illumination GLIs. Specifically, the color restoration of GLIs was realized based on modified dynamic threshold. After color correction, the low illumination GLI were restored and enhanced by a self-designed convolutional neural network (CNN). In this way, the authors achieved ideal effects of color restoration and clarity enhancement, while solving the difficulty of manual feature design in landscape design renderings. Finally, experiments were carried out to verify the feasibility and effectiveness of the proposed image color enhancement approach.

show abstract

Event Enhanced High-Quality Image Recovery

Cited by 84 publications

References 31 publications

EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation

EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation

Deep Learning for HDR Imaging: State-of-the-Art and Future Trends

Color Enhancement of Low Illumination Garden Landscape Images

Contact Info

Product

Resources

About