Linearizing the Plenoptic Space

Abstract

The plenoptic function, also known as the light field or the lumigraph, contains the information about the radiance of all optical rays that go through all points in space in a scene. Since no camera can capture all this information, one of the main challenges in plenoptic imaging is light field reconstruction, which consists in interpolating the ray samples captured by the cameras to create a dense light field. Most existing methods perform this task by first attempting some kind of 3D reconstruction of the visible scene. Our method, in contrast, works by modeling the scene as a set of visual points, which describe how each point moves in the image when a camera moves. We compute visual point models of various degrees of complexity, and show that highdimensional models are able to replicate complex optical effects such as reflection or refraction, and a model selection method can differentiate quasi-Lambertian from non-Lambertian areas in the scene.

Light Field Rendering for non-Lambertian Objects

“…The depth maps have been computed by stereo matching between the input image and an adjacent image with a disparity of approximately 10 pixels in 1000 × 1000 resolution. Linear maps (similarly to Nieto et al [23]) are also computed using stereo matching, but they encode the derivative for horizontal and vertical camera displacements. Thus, they are based on the retrieved correspondences between three images (reference, horizontal and vertical displacement).…”

“…If small specular reflections may remain unnoticed even if decreasing the photo-realism, reconstruction of fully refractive or reflective objects fails. Some recent advances in those techniques are more robust to this problem by taking into account the viewing direction [19,20] or the disparity of non-Lambertian features [23,22]. Other techniques focus specifically on transparent and refractive objects.…”

View Synthesis Tool for VR Immersive Video

“…When such objects, socalled non-Lambertian, are present in the scene, the linear hypothesis in pixel displacement in the function of the camera displacement is not valid anymore. Adapting the DIBR principles to non-Lambertian objects is nevertheless possible by exploiting additional information, such as structure, normal, and indexes of refraction [28], or a more accurate approximation of the pixel displacement [29][30][31] (chosen solution in RVS).…”

“…Alternatively, to model the non-Lambertian surface in itself, it is possible to track its feature movements on the surface [29,33,34]. DIBR can be generalized to non-Lambertian objects by replacing the usual depth maps with the coefficients of a polynomial approximating the non-Lambertian features displacement [30,31].…”

Polynomial Image-Based Rendering for non-Lambertian Objects

“…Applying the shearlet transform [23] on epipolar plane images (EPI) allows to render scenes with non-Lambertian objects and semi-transparency, by implicitly segmenting them [24], but does not approximate curved feature paths in the 4D light field. Indeed, non-Lambertian features are not constrained to a plane, hence, even locally, they have to be described with more complex models, using the general linear cameras approximation [25], [26], a local linear approximation [27] or global approximation with Bezier curves [28]. Multiplane Images (MPI) rendering approximates the straight lines visible in EPI by segmenting the scene in layered depth [29], [30] and seems to be resistant to artifacts created by non-Lambertian objects present in the scene [6].…”