Free segmentation in rendered 3D images through synthetic impulse response in integral imaging

Abstract

Integral Imaging is a technique that has the capability of providing not only the spatial, but also the angular information of three-dimensional (3D) scenes. Some important applications are the 3D display and digital post-processing as for example, depth-reconstruction from integral images. In this contribution we propose a new reconstruction method that takes into account the integral image and a simplified version of the impulse response function (IRF) of the integral imaging (InI) system to perform a two-dimensional (2D) deconvolution. The IRF of an InI system has a periodic structure that depends directly on the axial position of the object. Considering different periods of the IRFs we recover by deconvolution the depth information of the 3D scene. An advantage of our method is that it is possible to obtain nonconventional reconstructions by considering alternative synthetic impulse responses. Our experiments show the feasibility of the proposed method.Integral imaging is a technique based on the original idea proposed by Lippmann in 1908 [1]. To record a set of different perspectives of a 3D scene, i.e. a set of elemental images (EIs), a microlens array (MLA) is placed in front of a camera sensor. Because each microlens separates the rays to impact into different positions on the sensor array, both the spatial and also de angular information of the scene are captured by the system [2,3]. The original application of InI was related with the autostereoscopic display, and has been intensively developed during last years to produce a 3D image that can be visualized without the need for any special glasses [4][5][6]. However, display is not the only important application of InI technique [7][8][9][10][11][12]. The recover of the depth information of a 3D scene can be achieved from a single capture [13,14] by computationally projecting every EI through a virtual pinhole array or, equivalently overlapping and summing the intensities of the different elemental images [14]. Based on this principle, many digital reconstruction algorithms have been proposed [15][16][17][18][19][20]. Here, we have developed a new free depth-reconstruction method based on 2D deconvolution between the integral image capture in an InI setup and the corresponding IRF. By changing the period of the impulse response it is possible to select the appropriate depth of the reconstructed images. We also show that one can perform alternative reconstructions by considering non-conventional impulse responses.Let us start by describing the capture process of an InI system. As we can see from Fig. 1, each microlens of the MLA images the 3D object providing a 2D image on the CCD plane. Consider only one point of the 3D object. From Fig. 1 it is easy to see that the image recorded by the sensor camera has a periodic structure, so we can express the impulse response function (IRF) of the system as a comb function of period p(z) , beingNote that the period of the IRF is related with the axial position z of the point source and therefore, the further the ob...