A Calibration Method for Subaperture Views of Plenoptic 2.0 Camera Arrays

Abstract

We present a novel methodology to precisely calibrate the subaperture views of an array of plenoptic 2.0 cameras. Such cameras consist of a micro lens array, and the image captured through them is a lenslet image that can be converted to a dense set of pinhole views, the so-called subaperture images. This camera array provides several dense multiview images at some sparse points of 3D space. To find the relative position of those views, simply using structure-from-motion creates misalignments due to the small disparities within each set. Additionally, a traditional calibration using calibration patterns will also fail due to the complicated objectives of plenoptic 2.0 cameras and artifacts when they are converted to subaperture views. In this paper, we propose two calibration steps (a) to register the sparse central subaperture views using Structure-from-Motion which makes it robust to artifacts in the subaperture views, and (b) to register all dense multiview sets per plenoptic camera using camera's lenses specifications, disparity and distance to the scene. These two steps are followed by a novel merging process of the former registrations, to achieve precise calibration parameters for all the subaperture views of the multi-plenoptic array. Experimental results objectively and subjectively demonstrate high accuracy of the calibration. We show a 10% smaller reprojection error than using a naive structure-from-motion approach and verify that our method is suitable for high precision view synthesis applications such as virtual reality and holography.

Digital Media Design for Dynamic Gesture Interaction with Image Processing

“…• Additionally, since there is a major difference between the optical structure of plenoptic 1.0 and 2.0 cameras, we need to pay attention to their pixel intensity distributions, which helps us to design a more effective search algorithm that is suitable for each type of camera. In short, plenoptic 2.0 video coding is a new topic that has received significant attention from MPEG-I [26]- [30], [56]- [57]. However, it still needs further investigation in order to design an effective coding method to efficiently compress these kinds of data according to different acquisition devices.…”

“…While a comprehensive survey on coding solutions for plenoptic images can be found in [21]- [22], it is noted that plenoptic video coding has become a new research topic, receiving significant attention from MPEG-Immersive [23]- [25] recently. More importantly, since the commercial plenoptic 1.0 camera (Lytro Illum) is currently out of the market, the research community is focusing more on plenoptic 2.0 cameras (Raytrix) [26]- [30], which can provide more flexible control of the tradeoff between spatial-angular resolutions. Therefore, by considering the distinct intensity distribution of plenoptic 2.0 images, several new intra-coding tools [8]- [9] have been introduced that are designed and targeted for encoding plenoptic 2.0 image.…”

Multiview from micro-lens image of multi-focused plenoptic camera

“…For the view synthesis experiment, the cameras parameters are derived from the Plenoptic 2.0 image specifications (min/max object distance X n /X f , measured min/max disparity D min /D max , main lens focal length F , and micro-lens image pixel size p), following the method presented in MPEG-I lenslet video coding [14,19]. The baseline length between two sub-aperture images is given by:…”

Assessment of multi-plenoptic 2.0 camera depth maps for DIBR

“…11 Despite the current trend of improving LF processing, to the best of our knowledge, the state-of-the-art has not directly investigated the suitability of multi-plenoptic 2.0 cameras for DIBR applications. Significant efforts have been made to improve the quality of the depth maps, 12 a novel calibration strategy, 13 and a new method for converting plenoptic images into multiview images; 14 the MPEG-I Reference Lenslet content Converter (RLC). 15 Thus, we observe a lack of research on using plenoptic cameras for immersive applications.…”