Bridging the Gap Between Computational Photography and Visual Recognition

VidalMata, Rosaura G.; Banerjee, Sreya; RichardWebster, Brandon; Albright, Michael; Davalos, Pedro; McCloskey, Scott; Miller, Ben; Tambo, Asong; Ghosh, Sushobhan; Nagesh, Sudarshan; Yuan, Ye; Hu, Yueyu; Wu, Junru; Yang, Wenhan; Zhang, Xiaoshuai; Liu, Jiaying; Wang, Zhangyang; Chen, Hwann-Tzong; Huang, Tzu-Wen; Chin, Wen-Chi; Li, Yi-Chun; Lababidi, Mahmoud; Otto, Claudia; Scheirer, Walter J.

doi:10.1109/tpami.2020.2996538

Cited by 34 publications

(15 citation statements)

References 101 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Preprocessing for high-level vision tasks. As suggested in [49,42,8], SID has become a frequently-used preprocessing step, so we further examine whether our method bring benefits to downstream high-level vision tasks. Here, we randomly select 100 real-world hazy images from RTTS [20] with object categories and bounding boxes.…”

Section: Other Applicationsmentioning

confidence: 99%

Unpaired Deep Image Dehazing Using Contrastive Disentanglement Learning

Chen¹,

Fan²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present an effective unpaired learning based image dehazing network from an unpaired set of clear and hazy images. This paper provides a new perspective to treat image dehazing as a two-class separated factor disentanglement task, i.e, the task-relevant factor of clear image reconstruction and the task-irrelevant factor of haze-relevant distribution. To achieve the disentanglement of these two-class factors in deep feature space, contrastive learning is introduced into a CycleGAN framework to learn disentangled representations by guiding the generated images to be associated with latent factors. With such formulation, the proposed contrastive disentangled dehazing method (CDD-GAN) first develops negative generators to cooperate with the encoder network to update alternately, so as to produce a queue of challenging negative adversaries. Then these negative adversaries are trained end-to-end together with the backbone representation network to enhance the discriminative information and promote factor disentanglement performance by maximizing the adversarial contrastive loss. During the training, we further show that hard negative examples can suppress the task-irrelevant factors and unpaired clear exemples can enhance the task-relevant factors, in order to better facilitate haze removal and help image restoration. Extensive experiments on both synthetic and real-world datasets demonstrate that our method performs favorably against existing stateof-the-art unpaired dehazing approaches.

show abstract

Section: Other Applicationsmentioning

confidence: 99%

Unpaired Deep Image Dehazing Using Contrastive Disentanglement Learning

Chen¹,

Fan²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…As pointed out by a plenty of recent works (Wang et al 2016;Liu et al , 2019Liu et al , 2020Scheirer et al 2020;Yang et al 2020;Hahner et al 2019), the performance of high-level computer vision tasks, such as object detection and recognition, will deteriorate in the presence of various sensory and environmental degradation. In particular, Sakaridis et al (2018) studied the effect of image dehazing on semantic segmentation by a synthesized Foggy Cityscapes dataset with 20,550 images.…”

Section: Task-driven Evaluation Setsmentioning

confidence: 99%

A Comprehensive Benchmark Analysis of Single Image Deraining: Current Challenges and Future Perspectives

et al. 2021

Self Cite

View full text Add to dashboard Cite

The capability of image deraining is a highly desirable component of intelligent decision-making in autonomous driving and outdoor surveillance systems. Image deraining aims to restore the clean scene from the degraded image captured in a rainy day. Although numerous single image deraining algorithms have been recently proposed, these algorithms are mainly evaluated using certain type of synthetic images, assuming a specific rain model, plus a few real images. It remains unclear how these algorithms would perform on rainy images acquired "in the wild" and how we could gauge the progress in the field. This paper aims to bridge this gap. We present a comprehensive study and evaluation of existing single image deraining algorithms, using a new large-scale benchmark consisting of both synthetic and real-world rainy images of various rain types. This dataset highlights diverse rain models (rain streak, rain drop, rain and mist), as well as a rich variety of evaluation criteria (full-and no-reference objective, subjective, and task-specific). We further provide a comprehensive suite of criteria for deraining algorithm evaluation, including full-and no-reference metrics, subjective evaluation, and the novel task-driven evaluation. The proposed benchmark is accompanied with extensive experimental results that facilitate the assessment of the state-of-the-arts on a quantitative basis. Our evaluation and analysis indicate the gap between the achievable performance on synthetic rainy images and the practical demand on real-world images. We show that, despite many advances, image deraining is still a largely open problem. The paper is concluded by summarizing our general observations, identifying open research challenges and pointing out future directions. Our code and dataset is publicly available at http://uee.me/ddQsw. Keywords Image deraining • Image quality assessment • Deep convolution network • Benchmark analysis Communicated by Torsten Sattler.

show abstract

“…Following the success of the UG 2 challenge on this topic held at IEEE/CVF CVPR 2018 [41,42], a new challenge with an emphasis on video was organized at CVPR 2019. The UG 2 + 2019 Challenge provided an integrated forum for researchers to evaluate recent progress in handling various adverse visual conditions in real-world scenes, in robust, effective and task-oriented ways.…”

Section: * Denotes Equal Contributionmentioning

confidence: 99%

“…The main goal of this track is to correct visual aberrations present in video in order to improve the classification results obtained with out-of-the-box classification algorithms. For this we adapted the evaluation method and metrics provided in [42] to take into account the temporal factor of the data present in the UG 2 dataset. Below we introduce the adapted training and testing datasets, as well as the evaluation metrics and baseline classification results for this task 2 .…”

Section: Object Classification Improvement In Videomentioning

confidence: 99%

Report on UG^2+ Challenge Track 1: Assessing Algorithms to Improve Video Object Detection and Classification from Unconstrained Mobility Platforms

Banerjee¹,

VidalMata²,

Wang³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

How can we effectively engineer a computer vision system that is able to interpret videos from unconstrained mobility platforms like UAVs? One promising option is to make use of image restoration and enhancement algorithms from the area of computational photography to improve the quality of the underlying frames in a way that also improves automatic visual recognition. Along these lines, exploratory work is needed to find out which image pre-processing algorithms, in combination with the strongest features and supervised machine learning approaches, are good candidates for difficult scenarios like motion blur, weather, and mis-focus -all common artifacts in UAV acquired images. This paper summarizes the protocols and results of Track 1 of the UG 2 + Challenge held in conjunction with IEEE/CVF CVPR 2019. The challenge looked at two separate problems: (1) object detection improvement in video, and (2) object classification improvement in video. The challenge made use of the UG 2 (UAV, Glider, Ground) dataset, which is an established benchmark for assessing the interplay between image restoration and enhancement and visual recognition. 16 algorithms were submitted by academic and corporate teams, and a detailed analysis of how they performed on each challenge problem is reported here.

show abstract

Bridging the Gap Between Computational Photography and Visual Recognition

Cited by 34 publications

References 101 publications

Unpaired Deep Image Dehazing Using Contrastive Disentanglement Learning

Unpaired Deep Image Dehazing Using Contrastive Disentanglement Learning

A Comprehensive Benchmark Analysis of Single Image Deraining: Current Challenges and Future Perspectives

Report on UG^2+ Challenge Track 1: Assessing Algorithms to Improve Video Object Detection and Classification from Unconstrained Mobility Platforms

Contact Info

Product

Resources

About