Enhance Visual Recognition Under Adverse Conditions via Deep Networks

Abstract

Visual recognition under adverse conditions is a very important and challenging problem of high practical value, due to the ubiquitous existence of quality distortions during image acquisition, transmission, or storage. While deep neural networks have been extensively exploited in the techniques of low-quality image restoration and high-quality image recognition tasks respectively, few studies have been done on the important problem of recognition from very low-quality images. This paper proposes a deep learning based framework for improving the performance of image and video recognition models under adverse conditions, using robust adverse pre-training or its aggressive variant. The robust adverse pre-training algorithms leverage the power of pre-training and generalizes conventional unsupervised pre-training and data augmentation methods. We further develop a transfer learning approach to cope with real-world datasets of unknown adverse conditions. The proposed framework is comprehensively evaluated on a number of image and video recognition benchmarks, and obtains significant performance improvements under various single or mixed adverse conditions. Our visualization and analysis further add to the explainability of results.

Effects of Image Degradation and Degradation Removal to CNN-Based Image Classification

“…In [13], special degradations of low image resolution was studied in the applications of face identification, digit recognition and font recognition. In [14], a CNNbased method was proposed for improving the recognition performance of low-quality images and video by using pretraining, data augmentation, and other strategies. In this paper, we c onduct an empirical study to comprehensively understand the effects of various degradations to the performance of CNN-based image classification and investigate whether the use of degraded image in training and a pre-processing of degradation removal are helpful for image classification, which have important guiding significance to future work.…”

Delving Into Robust Object Detection From Unmanned Aerial Vehicles: A Deep Nuisance Disentanglement Approach

“…[7] proposed a super-resolution algorithm using coupled dictionary learning to transfer the target region into high resolution to "augment" its visual appearance. [50,29,32] proposed to internally super-resolve the feature maps of small objects to make them resemble similar characteristics as large objects. SNIP [45] showed that CNNs were not naturally robust to the variations in object scales.…”

Advancing Image Understanding in Poor Visibility Environments: A Collective Benchmark Study

“…These two stage joint optimization methods achieve better performance than previous one-stage methods. In [40,142], the joint optimization pipeline for low-resolution recognition is examined. In [41,42], Liu et al discussed the impact of denoising for semantic segmentation and advocated their mutual optimization.…”

“…Yet it remains questionable whether restoration-based approaches would actually boost the visual understanding performance, as the restoration/enhancement step is not optimized towards the target task and may bring in misleading information and artifacts too. For example, a recent line of researches [8,[39][40][41][42][43][44][45][46][47][48] discuss on the intrinsic interplay relationship of low-level vision and high-level recognition/detection tasks, showing that their goals are not always aligned.…”