Video Super-Resolution Using Multi-Scale and Non-Local Feature Fusion

Abstract

Video super-resolution can generate corresponding to high-resolution video frames from a plurality of low-resolution video frames which have rich details and temporally consistency. Most current methods use two-level structure to reconstruct video frames by combining optical flow network and super-resolution network, but this process does not deeply mine the effective information contained in video frames. Therefore, we propose a video super-resolution method that combines non-local features and multi-scale features to extract more in-depth effective information contained in video frames. Our method obtains long-distance effective information by calculating the similarity between any two pixels in the video frame through the non-local module, extracts the local information covered by different scale convolution cores through the multi-scale feature fusion module, and fully fuses feature information using different connection modes of convolution cores. Experiments on different data sets show that the proposed method is superior to the existing methods in quality and quantity.

Depth video super-resolution using a high-speed time-of-flight sensor

“…14 We adapt the structure of the VSR-DUF neural network to perform video super-resolution and denoising of SPAD dToF data. 12 This network comprises blocks of 3D convoluitions that extract spatio-temporal features without the need of extra steps (e.g. frame realignment).…”

“…11 Other techniques without alignment feature 3D convolution or recurrent neural networks to exploit spatio-temporal information directly. 12 Video super-resolution schemes have been designed for RGB data and they have not yet been optimised for the usage of depth frames. Some research has been conducted using depth, but these assume a static scene with a moving camera.…”