Directional Transforms for Video Coding Based on Lifting on Graphs

Abstract

Abstract-In this work we describe and optimize a general scheme based on lifting transforms on graphs for video coding. A graph is constructed to represent the video signal. Each pixel becomes a node in the graph and links between nodes represent similarity between them. Therefore, spatial neighbors and temporal motion-related pixels can be linked, while nonsimilar pixels (e.g., pixels across an edge) may not be. Then, a lifting-based transform, in which filterin operations are performed using linked nodes, is applied to this graph, leading to a 3-dimensional (spatio-temporal) directional transform which can be viewed as an extension of wavelet transforms for video. The design of the proposed scheme requires four main steps: (i) graph construction, (ii) graph splitting, (iii) filte design, and (iv) extension of the transform to different levels of decomposition. We focus on the optimization of these steps in order to obtain an effective transform for video coding. Furthermore, based on this scheme, we propose a coefficien reordering method and an entropy coder leading to a complete video encoder that achieves better coding performance than a motion compensated temporal filterin wavelet-based encoder and a simple encoder derived from H.264/AVC that makes use of similar tools as our proposed encoder (reference software JM15.1 configu ed to use 1 reference frame, no subpixel motion estimation, 16 × 16 inter and 4 × 4 intra modes).

Dense Light Field Coding: A Survey

“…For each nonoverlapping block of d ×d samples, a sparse adjacency matrix is constructed for each color component by connecting pixels based on the Euclidian distance between its nodes. Once the graph is constructed, a graph lifting transform [154] is applied. The transformed coefficients are uniformly quantized and entropy coded using an Amplitude and Group Partitioning (AGP) method [153].…”

Optimized Update/Prediction Assignment for Lifting Transforms on Graphs

“…The same idea was applied for image representation in [38] and for downsampling of signals on graphs [39]. In [40] we introduced a model-based UPA for a specific lifting-based video coding application. We considered a simplified model with only two different edge weights and did not prove optimality of the proposed UPA solution.…”

“…In Section III we formally define the UPA problem and describe our proposed general solution. In Section IV we analyze and 1 The topology is given by the geometry and the physics of the problem at hand (e.g., sensor networks or community graphs) or selected based on signal characteristics [35]- [37], [40]- [43], and the weights capture similarity between samples and can be defined, for example, as the inverse of the distance, using a Gaussian kernel weighting function [2], or inferred from the data [36], [37].…”

“…For the video example, we use the graph representation proposed in [35], [40], where every pixel is linked (i) to an arbitrary number of temporal neighbors (i.e., a pixel of frame in time instant t linked to a pixel in frame t + 1) following a motion estimation process and (ii) to its 8 one-hop spatial neighbors (i.e., pixels of the same frame) that do not cross contours 4 (we assume that pixels across contours will likely have very different luminance value). For every frame, weights have a different value for spatial and temporal links, and they are calculated following the process explained in [37], [40].…”

“…To obtain the graph at transformation level j from the graph at level j −1, we use the algorithm explained in detail in [40], Algorithm 2. Basically, we connect those U nodes that are either directly connected or at two-hop of distance in the graph at level j − 1, and the corresponding link weight is the product of the weights in the path between connected nodes at level j − 1.…”