Multiview from micro-lens image of multi-focused plenoptic camera

Abstract

Multi-focused Plenoptic cameras (Plenoptic 2.0) allow the acquisition of the Light-Field of a scene. However, extracting a novel view from the resulting Micro-Lens Array (MLA) image poses several challenges: micro-lenses calibration, noise reduction, patch size (depth) estimation to convert micro-lens image to multi-view images. We propose a novel method to easily find important micro-lenses parameters, avoid the unreliable luminance area, estimate the depth map, and extract sub-aperture images (multiview) for the single-and multifocused Plenoptic 2.0 camera. Our results demonstrate significant improvement in quality and reduction in computational time compared to the state-of-the-art conversion tool Reference Lenslet content Convertor from MLA image to multiview images.

Pattern-free Plenoptic 2.0 Camera Calibration

“…However, converting plenoptic 2.0 images to subaperture images is not straightforward and several methods exist. To test the independence of our method to the chosen conversion method, we compare the parameters obtained using three methods to render the subaperture images: Plenoptic toolbox (PT) [33] 1 , Reference Lenslet Convertor (RLC) [34]- [36] 2 and Lenslet to Multiview (LLMV) [37] 3 . They all need only the micro-images arrangement and diameter as a prior, but not the internal distances, focal length and focus distances.…”

“…Second, RLC is based on a Laplacian method [21]. Third, LLMV [37] is luminance-decay aware and extracts pixels for the subaperture images by estimating a per micro-image depth, smoothed and optimized using graph-cut. The last two software tools are used as reference tools by the MPEG dense light field community [36], [39].…”

“…Indeed, LLMV presents a smaller disparity range ([0, 1] pix) than the other methods ([0.5, 2.5] pix). Moreover, while PT and RLC show mostly blur artifacts, less impacting the depth estimation, LLMV is sensitive to patch artifacts [37], where the borders of the visible patches can distort the measured disparities.…”

“…1) Conversion of the plenoptic image to subaperture images; 2) Disparity estimation between those subaperture images; 3) Computation of the camera parameters using the camera specifications and the correspondences between disparity and depth values. Steps 1) and 2) can be performed using existing dedicated tools for subaperture image conversion [33]- [37] and disparity estimation [38]. For the third step, the next subsection describes the relations between the different parameters and the disparities.…”

Towards Non-Lambertian Scenes for Tensor Displays

“…where A (n) the tensor matricization, * is the Khatri-Rao product [13] and B (i) * is defined by Nagoya University's implementation [10] allows to output each layer from a light field using the extracted multi-views, which are an array of slightly different views that can be acquired by an array of cameras or the use of a plenoptic camera [14]. This method differs from the model as: (1) each view corresponds to a fixed direction, and (2) the gap between layers is constant.…”