A Fourier Disparity Layer Representation for Light Fields

Abstract

In this paper, we present a new Light Field representation for efficient Light Field processing and rendering called Fourier Disparity Layers (FDL). The proposed FDL representation samples the Light Field in the depth (or equivalently the disparity) dimension by decomposing the scene as a discrete sum of layers. The layers can be constructed from various types of Light Field inputs including a set of sub-aperture images, a focal stack, or even a combination of both. From our derivations in the Fourier domain, the layers are simply obtained by a regularized least square regression performed independently at each spatial frequency, which is efficiently parallelized in a GPU implementation. Our model is also used to derive a gradient descent based calibration step that estimates the input view positions and an optimal set of disparity values required for the layer construction. Once the layers are known, they can be simply shifted and filtered to produce different viewpoints of the scene while controlling the focus and simulating a camera aperture of arbitrary shape and size. Our implementation in the Fourier domain allows real time Light Field rendering. Finally, direct applications such as view interpolation or extrapolation and denoising are presented and evaluated.

Learning Fused Pixel and Feature-Based View Reconstructions for Light Fields

“…In Fig. 4(c), we evaluate the inherent capacity of our framework to extrapolate against two reference methods FDL [31] [9], EPI [11] or FDL [31], whereas 5 input views (grey) are used for LLFF [3]. and LLFF [3].…”

Light Field Compression Using Fourier Disparity Layers

“…The coding algorithm is depicted in Fig.1. First, the Fourier Disparity Layer calibration described in [8] determines a set of disparity values as well as the angular coordinates of each view. These parameters are needed for the FDL construction and view prediction steps, and are transmitted as metadata to the decoder.…”

“…A view Lu 0 of the light field at angular coordinate u 0 is then defined by Lu 0 pxq " Lpx, u 0 q. It has been shown in [8] that, given a set of n disparity values td k u kPv1,nw , the Fourier TransformLu 0 of Lu 0 can be decomposed as:…”

“…Typically, views that are close to the input views of the FDL construction step are better synthesized than more distant views. Furthermore, it was observed in [8] that constructing a FDL from all the outer views at the periphery of the light field is an ideal configuration for synthesizing the inner views. The circular prediction order illustrated in Fig.…”

“…In this paper, we describe a novel compression algorithm based on the Fourier Disparity Layer representation (FDL) introduced in [8]. This representation has been shown to be efficient for a variety of processing applications, namely rendering, view synthesis and denoising.…”