Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining

Abstract

Rain streaks can severely degrade the visibility, which causes many current computer vision algorithms fail to work. So it is necessary to remove the rain from images. We propose a novel deep network architecture based on deep convolutional and recurrent neural networks for single image deraining. As contextual information is very important for rain removal, we first adopt the dilated convolutional neural network to acquire large receptive field. To better fit the rain removal task, we also modify the network. In heavy rain, rain streaks have various directions and shapes, which can be regarded as the accumulation of multiple rain streak layers. We assign different alpha-values to various rain streak layers according to the intensity and transparency by incorporating the squeeze-and-excitation block. Since rain streak layers overlap with each other, it is not easy to remove the rain in one stage. So we further decompose the rain removal into multiple stages. Recurrent neural network is incorporated to preserve the useful information in previous stages and benefit the rain removal in later stages. We conduct extensive experiments on both synthetic and real-world datasets. Our proposed method outperforms the state-of-the-art approaches under all evaluation metrics. Codes and supplementary material are available at our project webpage: https://xialipku.github.io/RESCAN.

Confidence Measure Guided Single Image De-Raining

“…As it can be seen from this table, our method clearly out-performs the present state-ofthe-art image de-raining algorithms. On average, QuDeC outperformes the methods like RESCAN [28] and DIDMDN [1] by approximately 1dB. Furthermore, QuDeC outperformes the state-of-the-art method, UMRL+cycle-spinning [16], by 0.3dB on average.…”

“…We visually inspect the performance of different methods on real images, as we don't have the ground truth clean images. The performance of the proposed QuDeC method is compared against the following recent state-of-theart methods: (a) Fu et al [15] CNN method (TIP'17), (b) Joint Rain Detection and Removal (JORDER) [3] (CVPR17), (c) Deep detailed Network (DDN) [2] (CVPR'17), (d) Density-aware Image De-raining method using a Multistream Dense Network (DID-MDN) [1] (CVPR'18), (e) REcurrent SE Context Aggregation Net (RESCAN) [28] (ECCV'18) (f) Uncertainty guided Multi-scale Residual Learning (UMRL) network [16] (CVPR '19).…”

“…The DDN method is over de-raining on some images and on others it is slightly under de-raining as shown in the second column of Figure 11. First four rows show under de-raining performance corresponding to methods RESCAN [28] and DIDMDN [1] in the third and fourth columns of Figure 11 respectively where we can observe residue rain streaks in their outputs. The last three rows show over de-raining of methods RESCAN [28] and DIDMDN [1] in the third and the fourth columns of Figure 11, respectively where we can observe the edges of the objects are blurred and objects like cables or wires and texture of birds feather have disappeared.…”

“…Note that this method was specifically designed for removing rain drops from a glass rather than removing rain streaks from an image. [26], [27], [28] illustrated the importance of attention based methods in low-level vision tasks. In a recent work [28], Li et al proposed a convolutional and recurrent neural network-based method for single image de-raining which makes use of the contextual information for rain removal.…”

A Coarse-to-Fine Multi-stream Hybrid Deraining Network for Single Image Deraining

“…Removing rain streaks from images still remains an important and challenging topic in the outdoor tasks of computer vision and data mining [26,27] [30,31] [42,43,44], e.g., self-driving, drone-based video surveillance and real-time object recognition under severe rain weather conditions, etc. Since rain is one of the most common weather condition degrading the quality of images, but due to inappropriate expressions of images with rain streaks, the subsequent high-level tasks such as object detection [1], image recognition [8] and saliency detection [19] may be affected, so it is important to develop novel and effective models to remove rain streaks from images automatically.…”