Semi-MoreGAN: A New Semi-supervised Generative Adversarial Network for Mixture of Rain Removal

Abstract

Rain is one of the most common weather which can completely degrade the image quality and interfere with the performance of many computer vision tasks, especially under heavy rain conditions. We observe that: (i) rain is a mixture of rain streaks and rainy haze; (ii) the scene depth determines the intensity of rain streaks and the transformation into the rainy haze; (iii) most existing deraining methods are only trained on synthetic rainy images, and hence generalize poorly to the real-world scenes. Motivated by these observations, we propose a new SEMI-supervised Mixture Of rain REmoval Generative Adversarial Network (Semi-MoreGAN), which consists of four key modules: (i) a novel attentional depth prediction network to provide precise depth estimation; (ii) a context feature prediction network composed of several well-designed detailed residual blocks to produce detailed image context features; (iii) a pyramid depth-guided non-local network to effectively integrate the image context with the depth information, and produce the final rain-free images; and (iv) a comprehensive semi-supervised loss function to make the model not limited to synthetic datasets but generalize smoothly to realworld heavy rainy scenes. Extensive experiments show clear improvements of our approach over twenty representative state-of-the-arts on both synthetic and real-world rainy images. Code is available at https://github.com/syy-whu/Semi-MoreGAN.

Rain Streaks Removal in Images using Extended Generative Adversarial-based Deraining Framework

“…Due to the large light scattering and motion velocities, the raindrops generally produce bright streaks in the captured images or videos through cameras [2]. Such conditions can highly influence the image"s visual quality and also degrades outdoor vision systems" efficiency [3]. Also, it affects the efficacy of several computational vision mechanisms like event detection, scene analysis, and object detection and action recognition.…”