Not All Areas Are Equal: A Novel Separation‐Restoration‐Fusion Network for Image Raindrop Removal

Abstract

Detecting and removing raindrops from an image while keeping the high quality of image details has attracted tremendous studies, but remains a challenging task due to the inhomogeneity of the degraded region and the complexity of the degraded intensity. In this paper, we get rid of the dependence of deep learning on image-to-image translation and propose a separationrestoration-fusion network for raindrops removal. Our key idea is to recover regions of different damage levels individually, so that each region achieves the optimal recovery result, and finally fuse the recovered areas. In the region restoration module, to complete the restoration of a specific area, we propose a multi-scale feature fusion global information aggregation attention network to achieve global to local information aggregation. Besides, we also design an inside and outside dense connection dilated network, to ensure the fusion of the separated regions and the fine restoration of the image. The qualitatively and quantitatively evaluations are conducted to evaluate our method with the latest existing methods. The result demonstrates that our method outperforms state-of-the-art methods by a large margin on the benchmark datasets in extensive experiments. CCS Concepts • Computing methodologies → Image processing;

Semi‐MoreGAN: Semi‐supervised Generative Adversarial Network for Mixture of Rain Removal

“…Thereafter, many efforts have been made to either introduce advanced network modules and structures, or integrate problem‐related knowledge into network design. Network modules, such as dense block [FWF*18], recursive block [RZH*19] and dilated convolution [LWL*18,YTF*17,LQS*19,RLHS20b,RLHS20a,DWW*20], and structures, such as RNN [LWL*18, RZH*19], GAN [LCT19, ZSP19] and multi‐stream networks [YTF*17, LWL*18], are validated to be effective in rain streaks removal. Auxiliary information, including rain density [ZP18b], streak position [YTF*17] and gradient information [WZL*19], are leveraged to improve the robustness and performance of rain removal networks.…”

A Review of Detection and Removal of Raindrops in Automotive Vision Systems

“…Ren et al [ 73 ] proposed a raindrop removal system that was based on the proven “divide and concur” approach to solving complex problems. The process starts with separating the rainy image into different segments based on the amount of image distortion caused by raindrops.…”

“…This is followed by a median filter to reduce noise and dilation to smooth the edges. The separation process concludes by applying image binarization on the filtered and diluted mask, to generate Highly damaged and lightly damaged regions [ 73 ].…”

“…The objective function of the network is given by: where is the sum of smooth MAE between the restored image regions and target region images, and is distance loss between regions-fused image and the target image. are determined experimentally [ 73 ].…”

“… Structure of raindrop removal system with Separation-Restoration-Fusion approach, as proposed by Ren et al [ 73 ]. …”