Conditional Variational Image Deraining

Du, Yingjun; Xu, Jun; Zhen, Xiantong; Cheng, Ming–Ming; Shao, Ling

doi:10.1109/tip.2020.2990606

Cited by 79 publications

(29 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For its effectiveness a multi-stage training approach is followed. In [43] and [44] variational autoencoder based approach is proposed so as to use its' generative ability. A density estimation method in order to estimate the density map is used based on the fact that the rain density varies spatially and channel-wise.…”

Section: B Data-driven Methodsmentioning

confidence: 99%

Recursive Multi-Scale Image Deraining With Sub-Pixel Convolution Based Feature Fusion and Context Aggregation

et al. 2020

View full text Add to dashboard Cite

Along with several other low-vision based computer vision problems, single image deraining is also taken as a challenging one due to its ill-posedness. Several algorithms based on convolutional neural networks are devised that are either too simple to provide acceptable deraining results due to under-deraining or have a complex architectures that may result into over-deraining. In this paper we propose a deraining algorithm that is capable of boosting the reconstruction/deraining quality without the problem of over or under-deraining. Along with the originally proposed network, two of its' lightweight versions with reduced computational costs are also devised. Basically, we propose a recursively trained architecture that has two major components: a front-end module and a refinement module. The front-end module is based on dense fusion of lower label features followed by sub-pixel convolutions (pixel shuffling based convolutions). To refine and generate the enhanced deraining results further, we cascade a refinement module to the frontend module using multi-scale Context Aggregation Network (CAN) which includes feature fusion and pixel shuffling based convolutions. We present the deraining results in terms of Structural Similarity Index (SSIM) and Peak Signal to Noise Ratio (PSNR) on several benchmarks and compare with current state-of-theart algorithms. With comprehensive experiments on both real-world and synthetic datasets and extensive ablation study, we demonstrate that our approach produces better results compared to existing methods. INDEX TERMS Single image deraining, multi-scale context aggregation, sub-pixel convolution, feature fusion.

show abstract

Section: B Data-driven Methodsmentioning

confidence: 99%

Recursive Multi-Scale Image Deraining With Sub-Pixel Convolution Based Feature Fusion and Context Aggregation

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The method is to learn a nonlinear mapping function from a rain map to a background image using paired data sets. DerainNet proposed by Fu et al [33] is the first known model to apply a deep learning approach to the image rain removal task, which only makes high frequency information to train the network.Du et al [34] considered that rain patterns are different at different spatial locations and color channels and proposed a conditional variational rain removal network, while a rain pattern density estimation module was proposed for the purpose of adaptive rain pattern density. for estimating the rain pattern density.Zhang Zhang et al [35] also designed a multi-stream density estimator from the perspective of density to realize the adaptive rain removal of the image.…”

Section: Derainingmentioning

confidence: 99%

SDNet: mutil-branch for single image deraining using swin

Tan¹,

Kong²,

Fan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Rain streaks degrade the image quality and seriously affect the performance of subsequent computer vision tasks, such as autonomous driving, social security, etc. Therefore, removing rain streaks from a given rainy images is of great significance. Convolutional neural networks(CNN) have been widely used in image deraining tasks, however, the local computational characteristics of convolutional operations limit the development of image deraining tasks. Recently, the popular transformer has global computational features that can further facilitate the development of image deraining tasks. In this paper, we introduce Swin-transformer into the field of image deraining for the first time to study the performance and potential of Swin-transformer in the field of image deraining. Specifically, we improve the basic module of Swin-transformer and design a three-branch model to implement single-image rain removal. The former implements the basic rain pattern feature extraction, while the latter fuses different features to further extract and process the image features. In addition, we employ a jump connection to fuse deep features and shallow features. In terms of experiments, the existing public dataset suffers from image duplication and relatively homogeneous background. So we propose a new dataset Rain3000 to validate our model. Therefore, we propose a new dataset Rain3000 for validating our model. Experimental results on the publicly available datasets Rain100L, Rain100H and our dataset Rain3000 show that our proposed method has performance and inference speed advantages over the current mainstream single-image rain streaks removal models.The source code will be available at https://github.com/H-tfx/SDNet.

show abstract

“…Latent variable model based stochastic prediction: With extra latent variable involved, the latent variable models [11,12,18,48] can be used to achieve predictive distribution estimation. Currently, latent variable models have been used in numerous tasks, such as semantic segmentation [25], natural language processing [4], depth estimation [40], deraining [6], image deblurring [43], saliency detection [29,64], etc. In this paper, we explore latent variable models for saliency divergence modeling with "random sampling" strategy.…”

Section: Related Workmentioning

confidence: 99%