Motion compensation and scalability in lifting-based video coding

IEEE Trans. Circuits Syst. Video Technol.

Cid-Sueiro

Díaz-de-María

et al. 2018

Abstract-In this work we describe and optimize a general scheme based on lifting transforms on graphs for video coding. A graph is constructed to represent the video signal. Each pixel becomes a node in the graph and links between nodes represent similarity between them. Therefore, spatial neighbors and temporal motion-related pixels can be linked, while nonsimilar pixels (e.g., pixels across an edge) may not be. Then, a lifting-based transform, in which filterin operations are performed using linked nodes, is applied to this graph, leading to a 3-dimensional (spatio-temporal) directional transform which can be viewed as an extension of wavelet transforms for video. The design of the proposed scheme requires four main steps: (i) graph construction, (ii) graph splitting, (iii) filte design, and (iv) extension of the transform to different levels of decomposition. We focus on the optimization of these steps in order to obtain an effective transform for video coding. Furthermore, based on this scheme, we propose a coefficien reordering method and an entropy coder leading to a complete video encoder that achieves better coding performance than a motion compensated temporal filterin wavelet-based encoder and a simple encoder derived from H.264/AVC that makes use of similar tools as our proposed encoder (reference software JM15.1 configu ed to use 1 reference frame, no subpixel motion estimation, 16 × 16 inter and 4 × 4 intra modes).

Section: B Related Workmentioning

confidence: 99%

Directional Transforms for Video Coding Based on Lifting on Graphs

IEEE Trans. Circuits Syst. Video Technol.

Cid-Sueiro

Díaz-de-María

et al. 2018

“…For image coding, several works have been proposed in this area [1], [2], [3], [4]. For video coding, some approaches have been developed in which the filtering directions follow motion trajectories (motioncompensated temporal filtering (MCTF)) [5], [6]. The lifting scheme [7], which allows to construct multiresolution signal representations in a simple way, has been used in some of these works [5], [6], [2].…”

Section: Introductionmentioning

confidence: 99%

Filter optimization and complexity reduction for video coding using graph-based transforms

2013 IEEE International Conference on Image Processing

Díaz-de-María

Cid-Sueiro

et al. 2013

The basis functions of lifting transform on graphs are completely determined by finding a bipartition of the graph and defining the prediction and update filters to be used. In this work we consider the design of prediction filters that minimize the quadratic prediction error and therefore the energy of the detail coefficients, which will give rise to higher energy compaction. Then, to determine the graph bipartition, we propose a distributed maximum-cut algorithm that significantly reduces the computational cost with respect to the centralized version used in our previous work. The proposed techniques show improvements in coding performance and computational cost as compared to our previous work.

“…In the latter, each frame is firstly wavelet transformed in the spatial domain, followed by MCTF. Focusing in the temporal domain, representative examples of MCTF implementations are [6] and [7], which use motion-compensated lifting steps to implement the temporal wavelet transform, filtering along a set of motion trajectories described by a specific motion model. These approaches can be described as "separable" in that spatial and temporal filtering are applied in separate steps.…”

Section: Introductionmentioning

confidence: 99%

Lifting Transforms on Graphs for Video Coding

2011 Data Compression Conference

Ortega

2011

We present a new graph-based transform for video signals using wavelet lifting. Graphs are created to capture spatial and temporal correlations in video sequences. Our new transforms allow spatial and temporal correlation to be jointly exploited, in contrast to existing techniques, such as motion compensated temporal filtering, which can be seen as "separable" transforms, since spatial and temporal filtering are performed separately. We design efficient ways to form the graphs and to design the prediction and update filters for different levels of the lifting transform as a function of expected degree of correlation between pixels. Our initial results are promising, with improvements in performance as compared to existing methods in terms of PSNR as a function of the percentage of retained coefficients of the transform.