Multi-channel Reconstruction of Video Sequences from Low-Resolution and Compressed Observations

Abstract

Abstract.A framework for recovering high-resolution video sequences from sub-sampled and compressed observations is presented. Compression schemes that describe a video sequence through a combination of motion vectors and transform coefficients, e.g. the MPEG and ITU family of standards, are the focus of this paper. A multichannel Bayesian approach is used to incorporate both the motion vectors and transform coefficients in it. Results show a discernable improvement in resolution in the whole sequence, as compared to standard interpolation methods.

A survey on super-resolution imaging

“…Many works ( [97][98][99], for examples) have been done to simultaneously reconstruct multiple high-resolution images at one time, as illustrated in Fig. 8.…”

“…Borman and Stevenson [97] proposed an algorithm to reconstruct multiple high-resolution images simultaneously by imposing temporal smoothness constraint on the prior image model, while Zibetti and Mayer [98] reduced its computational complexity by only considering the temporally smooth low-resolution frames in the observation model. Alvarez et al [99] proposed a multichannel SR approach for the compressed image sequence, which applies the sliding window to select a set of lowresolution frames to reconstruct another set (same number of frames) of high-resolution frames simultaneously. However, all these three algorithms require large memory storage, since these multiple high-resolution and low-resolution images are required available at the same time during the reconstruction process.…”

“…The existing SR video approaches can be classified into the following four categories: (i) the sliding-window-based SR video approach [93][94][95][96], (ii) the simultaneous SR video approach [97][98][99], (iii) the sequential SR video approach [100][101][102][103][104], and (iv) the learning-based SR video approach [105][106][107]. In the following, a review of each category is presented.…”

A state-space super-resolution approach for video reconstruction

“…To overcome this drawback, many works [20][21][22] have been done to simultaneously reconstruct multiple highresolution images, as conceptually illustrated in Fig. 1b.…”

“…Likewise, in the SR video case, an image sequence with a higher-resolution is produced from a group of adjacent lower-resolution uncompressed image frames [20][21][22] or compressed image frames [23]. A direct way to apply the SR image techniques on the SR video problem, as what have been done by most of the existing SR approaches for video signals, is based on the sliding-window-based Bayesian approach [6], as illustrated in Fig.…”