A Hierarchical Approach for Rain or Snow Removing in a Single Color Image

Wang, Yinglong; Liu, Shuaicheng; Chen, Chen; Zeng, Bing

doi:10.1109/tip.2017.2708502

Cited by 225 publications

(154 citation statements)

References 32 publications

Supporting

Mentioning

153

Contrasting

Order By: Relevance

“…Note the methods proposed in [8], and [35] showed the benefits of using multiscale networks for image de-raining. Recently, Wang et al [28] proposed a hierarchical approach based on estimating different frequency details of an image to get the de-rained image. The method proposed by Qian et al [22] generates attentive maps using the recurrent neural networks, and then uses the features from different scales to compute the loss for removing the rain drops on glasses.…”

Section: Background and Related Workmentioning

confidence: 99%

Uncertainty Guided Multi-Scale Residual Learning-Using a Cycle Spinning CNN for Single Image De-Raining

Yasarla

Patel

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

269

138

View full text Add to dashboard Cite

Single image de-raining is an extremely challenging problem since the rainy image may contain rain streaks which may vary in size, direction and density. Previous approaches have attempted to address this problem by leveraging some prior information to remove rain streaks from a single image. One of the major limitations of these approaches is that they do not consider the location information of rain drops in the image. The proposed Uncertainty guided Multi-scale Residual Learning (UMRL) network attempts to address this issue by learning the rain content at different scales and using them to estimate the final de-rained output. In addition, we introduce a technique which guides the network to learn the network weights based on the confidence measure about the estimate. Furthermore, we introduce a new training and testing procedure based on the notion of cycle spinning to improve the final de-raining performance. Extensive experiments on synthetic and real datasets to demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art methods. Code is available at: https://github.com/rajeevyasarla/ UMRL--using-Cycle-Spinning

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Uncertainty Guided Multi-Scale Residual Learning-Using a Cycle Spinning CNN for Single Image De-Raining

Yasarla

Patel

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

269

138

View full text Add to dashboard Cite

show abstract

“…Besides, Ding et al [23] designed a guided L 0 smoothing filter to obtain a coarse rain-free or snow-free image, and Li et al [24] utilized patch-based GMM priors to distinguish and remove rain from background in a single image. Wang et al [25] designed a 3-layer hierarchical scheme to classify the highfrequency part into rain/snow and non-rain/snow components. Gu et al [26] jointly analyzed sparse representation and synthesis sparse representation to encode background scene and rain streaks.…”

Section: Single Image Rain and Snow Removal Methodsmentioning

confidence: 99%

Video Rain Streak Removal by Multiscale Convolutional Sparse Coding

Xie

Zhao

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

194

168

View full text Add to dashboard Cite

Video rain/snow removal from surveillance videos is an important task in the computer vision community since rain/snow existed in videos can severely degenerate the performance of many surveillance system. Various methods have been investigated extensively, but most only consider consistent rain/snow under stable background scenes. Rain/snow captured from practical surveillance camera, however, is always highly dynamic in time with the background scene transformed occasionally. To this issue, this paper proposes a novel rain/snow removal approach, which fully considers dynamic statistics of both rain/snow and background scenes taken from a video sequence. Specifically, the rain/snow is encoded as an online multi-scale convolutional sparse coding (OMS-CSC) model, which not only finely delivers the sparse scattering and multi-scale shapes of real rain/snow, but also well encodes their temporally dynamic configurations by real-time ameliorated parameters in the model. Furthermore, a transformation operator imposed on the background scenes is further embedded into the proposed model, which finely conveys the dynamic background transformations, such as rotations, scalings and distortions, inevitably existed in a real video sequence. The approach so constructed can naturally better adapt to the dynamic rain/snow as well as background changes, and also suitable to deal with the streaming video attributed its online learning mode. The proposed model is formulated in a concise maximum a posterior (MAP) framework and is readily solved by the ADMM algorithm. Compared with the state-of-the-art online and offline video rain/snow removal methods, the proposed method achieves better performance on synthetic and real videos datasets both visually and quantitatively. Specifically, our method can be implemented in relatively high efficiency, showing its potential to real-time video rain/snow removal.Index Terms-multi-scale, convolutional sparse coding, rain/snow removal, online learning, alignment method.

show abstract

“…Note the methods proposed in [2], and [1] showed the benefits of using multi-scale networks for image de-raining. Recently, Wang et al [24] proposed a hierarchical approach based on estimating different frequency details of an image to get the de-rained image. The method proposed by Qian et al [25] generates attentive maps using the recurrent neural networks, and then uses the features from different scales to compute the loss for removing the rain drops on glasses.…”

Section: Background and Related Workmentioning

confidence: 99%

Confidence Measure Guided Single Image De-Raining

Yasarla

Patel

2020

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Single image de-raining is an extremely challenging problem since the rainy images contain rain streaks which often vary in size, direction and density. This varying characteristic of rain streaks affect different parts of the image differently. Previous approaches have attempted to address this problem by leveraging some prior information to remove rain streaks from a single image. One of the major limitations of these approaches is that they do not consider the location information of rain drops in the image. The proposed Image Quality-based single image Deraining using Confidence measure (QuDeC), network addresses this issue by learning the quality or distortion level of each patch in the rainy image, and further processes this information to learn the rain content at different scales. In addition, we introduce a technique which guides the network to learn the network weights based on the confidence measure about the estimate of both quality at each location and residual rain streak information (residual map). Extensive experiments on synthetic and real datasets demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art methods.

show abstract

A Hierarchical Approach for Rain or Snow Removing in a Single Color Image

Cited by 225 publications

References 32 publications

Uncertainty Guided Multi-Scale Residual Learning-Using a Cycle Spinning CNN for Single Image De-Raining

Uncertainty Guided Multi-Scale Residual Learning-Using a Cycle Spinning CNN for Single Image De-Raining

Video Rain Streak Removal by Multiscale Convolutional Sparse Coding

Confidence Measure Guided Single Image De-Raining

Contact Info

Product

Resources

About