A comprehensive survey: Image deraining and stereo‐matching task‐driven performance analysis

Abstract

Deraining has been attracting a lot of attention from researchers, and various methods have been proposed, especially deep-networks are widely adopted in recent years. Their structures and learning become more and more complicated and diverse, making it difficult to analyze the contributions and improvements. In this paper, a comprehensive review for current rain removal methods is first provided to show their contributions. Specifically, they are reviewed in terms of handing rain streaks and rain mist. Second, besides evaluating their rain removal ability, they are also evaluated in terms of their impact on subsequent stereo-matching task. To this end, a new deraining dataset is first prepared, called Rain-Kitti2012 and Rain-Kitti2015. They are created by adding rain part to clean image-pairs in Kitti2012 and Kitti2015. By then, nine state-of-the-art deraining methods are evaluated with full-reference and no-reference image quality assessment metrics. Furthermore, the blurriness and distortion types introduced during deraining are measured. Finally, three learning-based stereo matching methods are compared, and they take the outputs of deraining methods as inputs. It is further discussed how derained images influence the accuracy of stereo matching, which can provide some insight for jointly handling rain removal and stereo matching. 1: A comprehensive review for the current rain removal methods is provided. They are categorized into rain-streak-oriented and rain-mist-oriented approaches in terms of degradation type, and are categorized into model-driven and data-driven approaches in terms of methodology. 2: A new image deraining dataset is introduced, which is the first dataset that can be used to perform stereo-matching-driven evaluation for deraining methods. The dataset is created by adding rain part to clean images in KITTI2012 and KITTI2015. 3: We evaluate 9 deep learning based deraining methods with full-reference and no-reference metrics. In addition, the types of distortions produced by these methods are discussed and measured quantitatively. And, the impact of 9 deraining methods on the subsequent stereo matching task is evaluated, which can provide some insight on how to design stereo matching task-driven deraining methods.

Attention‐generative adversarial networks for simulating rain field

“…As shown in Figure 9, we kept r i = 1, n p = 5 unchanged while varying r d = [0.1, 0.5, 0.9], and evaluating the influence of the proportion of rain density in the rain field on SRFGNet's generation performance. In Figure 10, we varied the number of rain peaks (n p ) between [5,12,25] while keeping r d = 0.4, r i = 0.1 unchanged, and studied their impact on SRFGNet's performance. Finally, we randomly sampled a set of generated data from the generated samples, while keepingr i = 0.1, r d = 0.4, n p = 5 unchanged.…”

“…Among the diverse ranges of weather conditions, rain is a common factor responsible for hindering the effectiveness of high‐level computer vision applications, particularly in fields like video surveillance and autonomous driving [3, 4]. The removal of rain streaks from rainy images can greatly enhance the efficiency of these vision‐related tasks [5].…”

A Survey of Deep Learning-Based Image Restoration Methods for Enhancing Situational Awareness at Disaster Sites: The Cases of Rain, Snow and Haze

“…Previous surveys on the subject [49][50][51][52][53] all provide a similar categorization of the surveyed methods. In this survey paper, we particularly highlight the attributes of more recent methods; that is, methods that were published in the year 2020 onward.…”

An application of stereo matching algorithm based on transfer learning on robots in multiple scenes

“…These algorithms aim to calculate disparities, which represent the horizontal displacement of corresponding pixels in two rectified stereo pairs. Traditional methods often rely on prior knowledge of the image to construct a stereo matching function that enables the generation of a dense disparity map 4 .…”