Light Field Compression Using Fourier Disparity Layers

Abstract

In this paper, we present a compression method for light fields based on the Fourier Disparity Layer representation. This light field representation consists in a set of layers that can be efficiently constructed in the Fourier domain from a sparse set of views, and then used to reconstruct intermediate viewpoints without requiring a disparity map. In the proposed compression scheme, a subset of light field views is encoded first and used to construct a Fourier Disparity Layer model from which a second subset of views is predicted. After encoding and decoding the residual of those predicted views, a larger set of decoded views is available, allowing us to refine the layer model in order to predict the next views with increased accuracy. The procedure is repeated until the complete set of light field views is encoded. Following this principle, we investigate in the paper different scanning orders of the light field views and analyse their respective efficiencies regarding the compression performance.

Dense Light Field Coding: A Survey

“…It is seen that, in average, the proposed solution achieves bit savings of 48.1% compared to WaSP [158] and 27.8% compared to MuLE [125]. For coding LF images acquired using a multi-camera array, the proposed solution achieves 28% of bit savings compared to WaSP [158] and slightly better performance, mainly at low bit rates, than the lifting DWT-based solution [148] and the solution in [249] using transform-assisted synthesis.…”

“…Experimental results are shown for coding LF images from the EPFL, HCI 4D and Fraunhofer LF Datasets (see Table 1) and comparing the proposed solution to eight different LF coding solutions: i) the WaSP solution [158] as in JPEG Pleno VM 2.1 [127]; ii) the DCT-based solution MuLE [125] as in the JPEG Pleno VM 2.1 [127]; iii) the lifting DTW-based solution in [148]; iv) the GFT-based solution in [155]; v) a PVS-based solution using serpentine ordering (see Fig. 9b) and HEVC; vi) the solution in [249] using transform-assisted view synthesis (see Section III-D2); vii) the solution in [250] using learningbased synthesis (see Section III-D3); and viii) the solution in [49] also using learning-based synthesis. For coding lenslet LF images, the proposed solution is seen to significantly outperform the WaSP [158], MuLE [125] and the GFT-based solution [155], while it is outperformed at low bit rates by the solutions using learning-based synthesis in [49], [250].…”

“…It is shown that the proposed solution achieves significant coding gains only at low bit rates, but it underperforms the PVSbased HEVC solution at high bit rates. In [249], a LF coding solution based on Fourier Disparity Layer (FDL) decomposition [252] and the coding architecture shown in Fig. 26 is proposed.…”

The JPEG Pleno Light Field Coding Standard 4D-Transform Mode: How to Design an Efficient 4D-Native Codec

“…For each HEVC's Coding Unit block size, the two new intra prediction modes select different reference blocks according to the depth of the current block. In [34], a novel coding solution based on Fourier Disparity Layer representation is introduced, where the LF is partitioned into subsets of views (PIs). While the first subset is coded as a video sequence using HEVC [21], the remaining subsets are iteratively estimated using Fourier Disparity Layer representation and the prediction residual coded with HEVC; in this solution, both depth map estimation and view synthesis techniques play important roles.…”

Local Low Rank Approximation With a Parametric Disparity Model for Light Field Compression

“…Instead of explicitly using disparity for view prediction, one can also exploit signal priors. This is the case in [32], [33] where the authors exploit light field sparsity in the 4D Fourier domain to reconstruct the entire light field from a subset of views. The approach is assessed using the SHVC (Scalable HEVC-based Video Coding) coding framework with two layers.…”