GTA-Net: Gradual Temporal Aggregation Network for Fast Video Deraining

Abstract

Recently, the development of intelligent technology arouses the requirements of high-quality videos. Rain streak is a frequent and inevitable factor to degrade the video. Many researchers have put their energies into eliminating the adverse effects of rainy video. Unfortunately, how to fully utilize the temporal information from rainy video is still in suspense. In this work, to effectively exploit temporal information, we develop a simple but effective network, Gradual Temporal Aggregation Network (GTA-Net for short). To be specific, according to the temporal distance between rainy frames and the reference frame, we divide the rainy frames into different groups. A multi-stream coarse temporal aggregation module is first performed to aggregate different temporal information with equal status and importance. Then we design a singlestream fine temporal aggregation module to further fuse the integrated frames that maintain the different distances with the target frame. In this way of coarse-to-fine, we not only achieve superior performance, but also gain the surprising execution speed owing to abandon the time-consuming alignment operation. Plenty of experimental results demonstrate that our GTA-Net performs favorably compared to other stateof-the-art approaches. The meticulous ablation study further indicates the effectiveness of our designed GTA-Net.

A Survey of Deep Learning-Based Image Restoration Methods for Enhancing Situational Awareness at Disaster Sites: The Cases of Rain, Snow and Haze

“…Method Model Short Description Year DetailNet [17] ACM reduces mapping range; promotes HF details Residual-guide [55] ACM cascaded; residuals; coarse-to-fine NLEDN [56] ACM end-to-end, non-locally-enhanced; spatial correlation DID-MDN [20] ACM density-aware multi-stream densely connected CNN DualCNN [57] ACM estimation of structures and details Scale-free [54] HRMLL wavelet analysis DMTNet [58] ACM symmetry reduces complexity; multidomain translation UC-PFilt [ [87] Equation (A8) contextual dilated network; squeeze-and-excitation block Pyramid Derain [86] ACM Gaussian-Laplacian pyramid decomposition DRN [93] ACM multi-stage residual network with two residual blocks NCANet [88] Equation (A10) rain streaks as residuals sum; recurrent PRRNet [91] ACM stereo; semantic segmentation; multi-view fusion Recurrent GTA-Net [105] ACM multi-stream coarse; single-stream fine…”