A state-space super-resolution approach for video reconstruction

Abstract

The main objective of super-resolution video reconstruction is to make use of a set of low-resolution image frames to produce their respective counterparts with higher resolution. The conventional two-equation-based Kalman filter only considers the information from the previously reconstructed high-resolution frame and the currently observed low-resolution frame for producing each high-resolution frame. It has been observed that the information inherited in the previously observed low-resolution frame could be beneficial on the reconstruction of the super-resolution video. For that, an extra observation equation is incorporated into the framework of the conventional two-equation-based Kalman filtering in this paper to establish a three-equation-based state-space approach as a more generalized framework. The closed-form solution is mathematically derived, and extensive simulations using both artificially degraded and real-life image sequences are conducted to demonstrate its superior performance. Furthermore, a unified theoretical analysis is provided to analyze the relationship between the proposed framework and two existing super-resolution approaches, the sliding-window-based Bayesian estimation approach and the conventional two-equation-based Kalman filtering, respectively.

Super-resolution: a comprehensive survey

“…- [62], [63], [64], [225], [278], [387], [427] assume that a sequence of LR observations are blurred and downsampled versions of a respective sequence of HR images, i.e., they don't consider warping effect between LR images and their corresponding HR ones, instead they involve warping between the superresolved HR images. This, provides the possibility of using temporal information between consecutive frames of a video sequence.…”

“…Iterative Adaptive Filtering (IAF) algorithms [62] (1999), [63], [64], [156], [210], [225], [226], [227], [278], [334], [387], [427], [463] have been developed mainly for generating a super-resolved video from an LR video (video to video SR), and treat the problem as a state estimation problem and therefore propose considering the Kalman filter for this purpose. To do so, besides the observation equation of the Kalman filter (which is the same as in the imaging model of Eq.…”

A survey on super-resolution imaging

“…The existing SR video approaches can be classified into the following four categories: (i) the sliding-window-based SR video approach [93][94][95][96], (ii) the simultaneous SR video approach [97][98][99], (iii) the sequential SR video approach [100][101][102][103][104], and (iv) the learning-based SR video approach [105][106][107]. In the following, a review of each category is presented.…”

“…A three-equation-based state-space filtering is proposed by Tian and Ma [104], by incorporating an extra observation equation into the framework of the conventional two-equation-based Kalman filtering to build up a threeequation-based state-space model. In [104], a full mathematical derivation for arriving at a closed-form solution is provided, which exploits the information from the previously reconstructed high-resolution frame, the currently observed low-resolution frame as well as the previously observed low-resolution frame for producing the next high-resolution frame.…”

“…In [104], a full mathematical derivation for arriving at a closed-form solution is provided, which exploits the information from the previously reconstructed high-resolution frame, the currently observed low-resolution frame as well as the previously observed low-resolution frame for producing the next high-resolution frame. The above-mentioned steps will be sequentially processed across the image frames.…”