Semi-Supervised Transfer Learning for Image Rain Removal

Abstract

Single image rain removal is a typical inverse problem in computer vision. The deep learning technique has been verified to be effective for this task and achieved state-of-theart performance. However, previous deep learning methods need to pre-collect a large set of image pairs with/without synthesized rain for training, which tends to make the neural network be biased toward learning the specific patterns of the synthesized rain, while be less able to generalize to real test samples whose rain types differ from those in the training data. To this issue, this paper firstly proposes a semi-supervised learning paradigm toward this task. Different from traditional deep learning methods which only use supervised image pairs with/without synthesized rain, we further put real rainy images, without need of their clean ones, into the network training process. This is realized by elaborately formulating the residual between an input rainy image and its expected network output (clear image without rain) as a specific parametrized rain streaks distribution. The network is therefore trained to adapt real unsupervised diverse rain types through transferring from the supervised synthesized rain, and thus both the short-of-training-sample and bias-to-supervised-sample issues can be evidently alleviated. Experiments on synthetic and real data verify the superiority of our model compared to the state-of-the-arts.

Recursive Multi-Scale Image Deraining With Sub-Pixel Convolution Based Feature Fusion and Context Aggregation

“…Very few approaches are proposed considering the semi-supervised learning scheme which is very important in case of real-world image deraining due to the lack of paired rainy-clean images. Recently [46] and [47] propose a semi-supervised deraining techniques where the network is trained simultaneously with labelled and unlabelled data. In [46], for unsupervised loss, rain residual is modeled as a likelihood in a gaussian mixture model.…”

“…Recently [46] and [47] propose a semi-supervised deraining techniques where the network is trained simultaneously with labelled and unlabelled data. In [46], for unsupervised loss, rain residual is modeled as a likelihood in a gaussian mixture model. In [47], gaussian process based non-parametric approach is used where the intermediate latent space from the network is used to generate the pseudo ground truth for the unlabeled data.…”

Instance-Level Future Motion Estimation in a Single Image Based on Ordinal Regression and Semi-Supervised Domain Adaptation

“…Also, various computer vision applications have exploited semi-supervised learning methods to reduce expensive labeling efforts. They include 3D human pose estimation [50], 3D hand pose estimation [51], deraining [52], scene parsing [53], multi-view keypoint detection [54], ob-This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.…”

Model-Based Deep Network for Single Image Deraining

“…Although existing deep learning-based deraining methods have achieved significant performance improvement, they still have the following limitations: (1) Most of them use an inaccurate linear rain model. In [19]- [22], [27], a simplified linear additive rain model is adopted, which separates a rainy image into a linear combination of a clean background image and a rain map. (2) Some of them [23], [29], [30] are fully data-driven deep networks without embedding rain model into the networks, which usually have limitations in interpretability and generalization.…”