Rain Streaks Removal for Single Image via Kernel-Guided Convolutional Neural Network

Wang, Ye-Tao; Zhao, Xi-Le; Jiang, Tai-Xiang; Deng, Liang-Jian; Chang, Yi; Huang, Ting‐Zhu

doi:10.1109/tnnls.2020.3015897

Cited by 60 publications

(19 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…KGCNN [55] Idea: Rain streaks are removed in a stage by stage manner. The accumulation of rain streaks and the global atmospheric light are considered in rain model;…”

Section: Model-driven Methodsmentioning

confidence: 99%

A comprehensive survey: Image deraining and stereo‐matching task‐driven performance analysis

Liu

Zhao

et al. 2021

IET Image Processing

View full text Add to dashboard Cite

Deraining has been attracting a lot of attention from researchers, and various methods have been proposed, especially deep-networks are widely adopted in recent years. Their structures and learning become more and more complicated and diverse, making it difficult to analyze the contributions and improvements. In this paper, a comprehensive review for current rain removal methods is first provided to show their contributions. Specifically, they are reviewed in terms of handing rain streaks and rain mist. Second, besides evaluating their rain removal ability, they are also evaluated in terms of their impact on subsequent stereo-matching task. To this end, a new deraining dataset is first prepared, called Rain-Kitti2012 and Rain-Kitti2015. They are created by adding rain part to clean image-pairs in Kitti2012 and Kitti2015. By then, nine state-of-the-art deraining methods are evaluated with full-reference and no-reference image quality assessment metrics. Furthermore, the blurriness and distortion types introduced during deraining are measured. Finally, three learning-based stereo matching methods are compared, and they take the outputs of deraining methods as inputs. It is further discussed how derained images influence the accuracy of stereo matching, which can provide some insight for jointly handling rain removal and stereo matching. 1: A comprehensive review for the current rain removal methods is provided. They are categorized into rain-streak-oriented and rain-mist-oriented approaches in terms of degradation type, and are categorized into model-driven and data-driven approaches in terms of methodology. 2: A new image deraining dataset is introduced, which is the first dataset that can be used to perform stereo-matching-driven evaluation for deraining methods. The dataset is created by adding rain part to clean images in KITTI2012 and KITTI2015. 3: We evaluate 9 deep learning based deraining methods with full-reference and no-reference metrics. In addition, the types of distortions produced by these methods are discussed and measured quantitatively. And, the impact of 9 deraining methods on the subsequent stereo matching task is evaluated, which can provide some insight on how to design stereo matching task-driven deraining methods.

show abstract

“…KGCNN [55] Idea: Rain streaks are removed in a stage by stage manner. The accumulation of rain streaks and the global atmospheric light are considered in rain model;…”

Section: Model-driven Methodsmentioning

confidence: 99%

A comprehensive survey: Image deraining and stereo‐matching task‐driven performance analysis

Liu

Zhao

et al. 2021

IET Image Processing

View full text Add to dashboard Cite

show abstract

“…Some DL-based image restoration tasks, including dehazing, deraining, and desnowing, rarely use these priors. However, kernel estimation is an essential step in dehazing, deraining, and desnowing [204]. Consequently, a potential direction is to convert the estimated kernels to a novel type of priors for other deep image restoration and enhancement tasks.…”

Section: Potential Of Priors Based On Kernel Modelingmentioning

confidence: 99%

All One Needs to Know about Priors for Deep Image Restoration and Enhancement: A Survey

Lu¹,

Lin²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Image restoration and enhancement is a process of improving the image quality by removing degradations, such as noise, blur, and resolution degradation. Deep learning (DL) has recently been applied to image restoration and enhancement. Due to its ill-posed property, plenty of works have explored priors to facilitate training deep neural networks (DNNs). However, the importance of priors has not been systematically studied and analyzed by far in the research community. Therefore, this paper serves as the first study that provides a comprehensive overview of recent advancements of priors for deep image restoration and enhancement. Our work covers five primary contents: (1) A theoretical analysis of priors for deep image restoration and enhancement; (2) A hierarchical and structural taxonomy of priors commonly used in the DL-based methods; (3) An insightful discussion on each prior regarding its principle, potential, and applications; (4) A summary of crucial problems by highlighting the potential future directions to spark more research in the community; ( 5) An open-source repository that provides a taxonomy of all mentioned works and code links.

show abstract

“…Network modules, such as dense block Fan et al (2018); Wang et al (2019a), recursive block Ren et al (2019) and dilated convolution Deng et al (2020); Yang et al (2017), and structures, such as RNN Li et al (2018); Ren et al (2019), GAN Li et al (2019b) and multi-stream networks Deng et al (2020); Yang et al (2017), are validated to be effective in rain streak removal. Auxiliary information, including rain density Zhang and Patel (2018b), streak position Yang et al (2017), gradient information Wang et al (2019d) and motion blur kernel Wang et al (2018b), are leveraged to improve the robustness and performance of deraining networks. However, it is still challenging to collect paired real-world rainy images for training and hence, existing methods typically use synthetic rain datasets to training their models, which lead to sub-optimal performance on the real-world images due to the significant gap between synthetic and rainy images.…”

Section: Rain Streak Removalmentioning

confidence: 99%

Semi-MoreGAN: A New Semi-supervised Generative Adversarial Network for Mixture of Rain Removal

Yan¹,

Wang²,

Wei³

et al. 2022

Preprint

View full text Add to dashboard Cite

Rain is one of the most common weather which can completely degrade the image quality and interfere with the performance of many computer vision tasks, especially under heavy rain conditions. We observe that: (i) rain is a mixture of rain streaks and rainy haze; (ii) the scene depth determines the intensity of rain streaks and the transformation into the rainy haze; (iii) most existing deraining methods are only trained on synthetic rainy images, and hence generalize poorly to the real-world scenes. Motivated by these observations, we propose a new SEMI-supervised Mixture Of rain REmoval Generative Adversarial Network (Semi-MoreGAN), which consists of four key modules: (i) a novel attentional depth prediction network to provide precise depth estimation; (ii) a context feature prediction network composed of several well-designed detailed residual blocks to produce detailed image context features; (iii) a pyramid depth-guided non-local network to effectively integrate the image context with the depth information, and produce the final rain-free images; and (iv) a comprehensive semi-supervised loss function to make the model not limited to synthetic datasets but generalize smoothly to realworld heavy rainy scenes. Extensive experiments show clear improvements of our approach over twenty representative state-of-the-arts on both synthetic and real-world rainy images. Code is available at https://github.com/syy-whu/Semi-MoreGAN.

show abstract

Rain Streaks Removal for Single Image via Kernel-Guided Convolutional Neural Network

Cited by 60 publications

References 68 publications

A comprehensive survey: Image deraining and stereo‐matching task‐driven performance analysis

A comprehensive survey: Image deraining and stereo‐matching task‐driven performance analysis

All One Needs to Know about Priors for Deep Image Restoration and Enhancement: A Survey

Semi-MoreGAN: A New Semi-supervised Generative Adversarial Network for Mixture of Rain Removal

Contact Info

Product

Resources

About