A Novel Tensor-Based Video Rain Streaks Removal Approach via Utilizing Discriminatively Intrinsic Priors

Jiang, Tai-Xiang; Huang, Ting‐Zhu; Zhao, Xi-Le; Deng, Liang-Jian; Wang, Yao

doi:10.1109/cvpr.2017.301

Cited by 153 publications

(106 citation statements)

References 29 publications

Supporting

Mentioning

104

Contrasting

Unclassified

Order By: Relevance

“…To enhance the robustness of rain removal, Barnum et al [17] employed the regular visual effects of rain and snow in global frequency information to approximate rain streaks as a motionblurred Gaussian. Afterwards, to integrate more prior knowledge of the task, Jiang et al [18] proposed a tensor-based video rain streak removal approach by considering the sparsity of rain streaks, smoothness along the raindrops and the rain-perpendicular direction, and global and local correlation along time direction.…”

Section: Video Rain and Snow Removal Methodsmentioning

confidence: 99%

“…By contrast, the OTMS-CSC method, as well as MS-SCS, is capable of better removing all the rain streaks without mixing extra information into the rain layer. (e) Liu et al [13] (f) TMS-CSC (g) OTMS-CSC (a) Input (b) Garg et al [5] (c) Jiang et al [18] (d) Ren et al [11] (e) Liu et al [13] (f) TMS-CSC (g) OTMS-CSC Fig. 9 and Fig.…”

Section: Experiments On Videos With Real Rain/snowmentioning

confidence: 99%

“…We first introduce experiments executed on four videos with synthetic rain/snow, three with static backgrounds, as shown in Fig. 3 -5, and one with evidently dynamic background with (a) Input/GT (b) Garg et al [16] (c) Jiang et al [18] (d) Ren et al [11] (e) Wei et al [12] (f) Liu et al [13] (g) MS-CSC (h) OTMS-CSC (a) Input (b) GT (c) Garg et al [16] (d) Jiang et al [18] (e) Ren et al [11] (f) Wei et al [12] (g) Liu et al [13] (h) MS-CSC (i) OTMS-CSC [16] (c) Jiang et al [18] (d) Ren et al [11] (e) Wei et al [12] (f) Liu et al [13] (f) MS-CSC (g) OTMS-CSC (a) Input (b) Garg et al [5] (c) Jiang et al [18] (d) Ren et al [11] (e) Groundtruth (f) Liu et al [13] (g) TMS-CSC (h) OTMS-CSC sequences, respectively. The rain/snow with various types were synthetically generated by Photoshop on a black background.…”

Section: Experiments On Videos With Synthetic Rain/snowmentioning

confidence: 99%

“…5, it can be seen that most competing methods have not finely removed snow from the video, and the separated snow layer by MS-CSC method improperly contains certain moving objects. Comparatively, the OTMS-CSC method has a better [16] (c) Jiang et al [18] (d) Ren et al [11] (e) Wei et al [12] (f) Liu et al [13] (g) MS-CSC (h) OTMS-CSC (a) Input (b) Garg et al [16] (c) Jiang et al [18] (d) Ren et al [11] (e) Wei et al [12] (f) Liu et al [13] (g) MS-CSC (h) OTMS-CSC performance in both snow removing and background/foreground detail preservation.…”

Section: Experiments On Videos With Synthetic Rain/snowmentioning

confidence: 99%

“…https://www.youtube.com/watch? {v=KzEv1h-JgaY, v=kNTYEKjXqzs, v=wb3gWRcKyCI, v=HbgoKKj7TNA} (a) Input (b) Garg et al [16] (c) Jiang et al [18] (d) Ren et al [11] (e) Wei et al [12] (f) Liu et al [13] (g) MS-CSC (h) OTMS-CSC (e) Wei et al [12] (f) Liu et al [13] (g) MS-CSC (h) OTMS-CSC (a) Input (b) Garg et al [5] (c) Jiang et al [18] (d) Ren et al [11] (e) Liu et al [13] (f) TMS-CSC (g) OTMS-CSC The videos shown in Fig. 7 and Fig.…”

Section: Experiments On Videos With Real Rain/snowmentioning

confidence: 99%

See 4 more Smart Citations

Video Rain Streak Removal by Multiscale Convolutional Sparse Coding

Xie

Zhao

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

193

168

View full text Add to dashboard Cite

Video rain/snow removal from surveillance videos is an important task in the computer vision community since rain/snow existed in videos can severely degenerate the performance of many surveillance system. Various methods have been investigated extensively, but most only consider consistent rain/snow under stable background scenes. Rain/snow captured from practical surveillance camera, however, is always highly dynamic in time with the background scene transformed occasionally. To this issue, this paper proposes a novel rain/snow removal approach, which fully considers dynamic statistics of both rain/snow and background scenes taken from a video sequence. Specifically, the rain/snow is encoded as an online multi-scale convolutional sparse coding (OMS-CSC) model, which not only finely delivers the sparse scattering and multi-scale shapes of real rain/snow, but also well encodes their temporally dynamic configurations by real-time ameliorated parameters in the model. Furthermore, a transformation operator imposed on the background scenes is further embedded into the proposed model, which finely conveys the dynamic background transformations, such as rotations, scalings and distortions, inevitably existed in a real video sequence. The approach so constructed can naturally better adapt to the dynamic rain/snow as well as background changes, and also suitable to deal with the streaming video attributed its online learning mode. The proposed model is formulated in a concise maximum a posterior (MAP) framework and is readily solved by the ADMM algorithm. Compared with the state-of-the-art online and offline video rain/snow removal methods, the proposed method achieves better performance on synthetic and real videos datasets both visually and quantitatively. Specifically, our method can be implemented in relatively high efficiency, showing its potential to real-time video rain/snow removal.Index Terms-multi-scale, convolutional sparse coding, rain/snow removal, online learning, alignment method.

show abstract

Section: Video Rain and Snow Removal Methodsmentioning

confidence: 99%

Section: Experiments On Videos With Real Rain/snowmentioning

confidence: 99%

Section: Experiments On Videos With Synthetic Rain/snowmentioning

confidence: 99%

Section: Experiments On Videos With Synthetic Rain/snowmentioning

confidence: 99%

Section: Experiments On Videos With Real Rain/snowmentioning

confidence: 99%

See 3 more Smart Citations

Video Rain Streak Removal by Multiscale Convolutional Sparse Coding

Xie

Zhao

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

193

168

View full text Add to dashboard Cite

show abstract

Video smoke removal based on low‐rank tensor completion via spatial‐temporal continuity constraint

Zhu

Liu

et al. 2021

Concurrency and Computation

View full text Add to dashboard Cite

Smoke has a very bad effect on the outdoor vision system. Not only are the videos with poor visual effects obtained, but also the quality and structure of the videos are reduced. In this paper, we propose a video smoke removal method based on low‐rank tensor completion via spatial‐temporal continuity constraint. The proposed method is based on the smoke mixing model and consider the sparseness of smoke and the global and local consistency of clean video. Then, the optimal solution of the smoke removal algorithm model is quickly realized by the Alternating Direction Method of Multiplier. Finally, we evaluate the experiment results of real‐world data and simulated data from the visual effects and objective indicators. And the experiment results show that our proposed algorithm can achieve better smoke removal results.

show abstract

A review of video‐based rainfall measurement methods

Yan

Chen

et al. 2023

WIREs Water

View full text Add to dashboard Cite

Accurate and high spatiotemporal resolution rainfall observations are essential for hydrological forecasting and flood management, especially in urban hydrological applications. However, it is difficult for traditional rainfall gauges, weather radars, and satellites to accurately estimate rainfall while simultaneously capturing the spatial and temporal variability of rainfall well. In this context, video‐based rainfall measurement, a novel method, has the advantages of real‐time performance and low cost and may thus provide a new way to establish rainfall observation networks with high spatial and temporal resolution. In recent years, different algorithms have been developed to recognize raindrops and estimate rainfall from rainfall videos. It has been demonstrated that video‐based rainfall measurement methods can provide comprehensive rainfall information with fine spatial and temporal granularity. However, raindrop visibility and the depth of field effects are difficult to address. The motion blur effect of raindrops may result in substantial errors and uncertainties. A fundamental problem of video‐based rainfall measurements lies in locating raindrops and accurately calculating their actual size. Moreover, the effectiveness of deep learning‐based video rainfall measurement models is greatly influenced by the diversity of the training data. Therefore, enhancing the high robustness and accuracy of video‐based rainfall measurement algorithms and increasing the computational efficiency are paramount to further development, which are prerequisites for their application in practical rainfall monitoring and developing multicamera monitoring networks.This article is categorized under: Science of Water > Methods Science of Water > Hydrological Processes

show abstract

A Novel Tensor-Based Video Rain Streaks Removal Approach via Utilizing Discriminatively Intrinsic Priors

Cited by 153 publications

References 29 publications

Video Rain Streak Removal by Multiscale Convolutional Sparse Coding

Video Rain Streak Removal by Multiscale Convolutional Sparse Coding

Video smoke removal based on low‐rank tensor completion via spatial‐temporal continuity constraint

A review of video‐based rainfall measurement methods

Contact Info

Product

Resources

About