Geometry-Aware Graph Transforms for Light Field Compact Representation

Abstract

The paper addresses the problem of energy compaction of dense 4D light fields by designing geometry-aware local graph-based transforms. Local graphs are constructed on super-rays that can be seen as a grouping of spatially and geometry-dependent angularly correlated pixels. Both non separable and separable transforms are considered. Despite the local support of limited size defined by the super-rays, the Laplacian matrix of the non separable graph remains of high dimension and its diagonalization to compute the transform eigen vectors remains computationally expensive. To solve this problem, we then perform the local spatio-angular transform in a separable manner. We show that when the shape of corresponding super-pixels in the different views is not isometric, the basis functions of the spatial transforms are not coherent, resulting in decreased correlation between spatial transform coefficients. We hence propose a novel transform optimization method that aims at preserving angular correlation even when the shapes of the super-pixels are not isometric. Experimental results show the benefit of the approach in terms of energy compaction. A coding scheme is also described to assess the ratedistortion perfomances of the proposed transforms and is compared to state of the art encoders namely HEVC and JPEG Pleno VM 1.1.

Dense Light Field Coding: A Survey

“…Since the solution is outperformed by HEVC for low bit rates, the authors suggest using the solution for applications such as archiving and instant storage on lenslet cameras. In [155], a GFT solution with a support defined on super-ray segmentation is proposed for lenslet LF coding. As proposed in [156], a super-ray segmentation is used for grouping light rays of similar color values and being close spatially in the 3D space by taking into account the disparity information.…”

“…Therefore, as the encoder proceeds towards the higher hierarchical layers, the density of the reference views increases and the intermediate view prediction becomes more efficient [248]. Experimental results are shown for coding LF images from the EPFL, HCI 4D and Fraunhofer LF Datasets (see Table 1) and comparing the proposed solution to eight different LF coding solutions: i) the WaSP solution [158] as in JPEG Pleno VM 2.1 [127]; ii) the DCT-based solution MuLE [125] as in the JPEG Pleno VM 2.1 [127]; iii) the lifting DTW-based solution in [148]; iv) the GFT-based solution in [155]; v) a PVS-based solution using serpentine ordering (see Fig. 9b) and HEVC; vi) the solution in [249] using transform-assisted view synthesis (see Section III-D2); vii) the solution in [250] using learningbased synthesis (see Section III-D3); and viii) the solution in [49] also using learning-based synthesis.…”

Graph-Based Spatio-Angular Prediction for Quasi-Lossless Compression of Light Fields

“…To be well suited for local separable graph transforms, the super-rays need to be consistent across the views. We hence consider the approach described in [8] to first compute super-pixels in the top-left view, and then project, using the top left view disparity, its segmentation labels to all the other views. The occluded pixels are assigned the label of the neighboring super-ray corresponding to the foreground objects (i.e.…”

“…Therefore, the basis functions of each spatial graph transform U k,v are not the same in the different views v, resulting in decreased correlation of the spatial transform coefficientsx k,v across views. To circumvent this issue, we have proposed in [8], a basis optimization procedure to find the basis functions U k,v that approximately diagonalize their respective Laplacians L k,v while being coherent across the views, given the scene geometry. By coherence, we mean the similarity of the eigenvectors in different views along the epipolar lines of the light field.…”

“…which connects pixels both within super-pixels and the ray directions), remains intractable. We therefore consider separable transforms using the optimization procedure introduced in [8] to preserve angular correlation in the spatial transform domain. The local support however does not allow us to capture long term spatial dependencies, unlike efficient predictive schemes used in state of the art codecs as HEVC.…”