Bidirectional Temporal-Recurrent Propagation Networks for Video Super-Resolution

Abstract

Recently, convolutional neural networks have made a remarkable performance for video super-resolution. However, how to exploit the spatial and temporal information of video efficiently and effectively remains challenging. In this work, we design a bidirectional temporal-recurrent propagation unit. The bidirectional temporal-recurrent propagation unit makes it possible to flow temporal information in an RNN-like manner from frame to frame, which avoids complex motion estimation modeling and motion compensation. To better fuse the information of the two temporal-recurrent propagation units, we use channel attention mechanisms. Additionally, we recommend a progressive up-sampling method instead of one-step up-sampling. We find that progressive up-sampling gets better experimental results than one-stage up-sampling. Extensive experiments show that our algorithm outperforms several recent state-of-the-art video super-resolution (VSR) methods with a smaller model size.

A lightweight hardware‐efficient recurrent network for video super‐resolution

“…Consequently, the back‐propagation of hidden states is avoided, and the latency caused by caching LR t + 1 is also reduced. In NetH, we introduce two SRA blocks, which employ residual learning and channel attention mechanism [6]. The detailed structure of the SRA block is illustrated in Figure 3.…”