Smooth Kernel Density Estimate for Multiple View Reconstruction

Abstract

We present a statistical framework to merge the information from silhouettes segmented in multiple view images to infer the 3D shape of an object. The approach is generalising the robust but discrete modelling of the visual hull by using the concept of averaged likelihoods. One resulting advantage of our framework is that the objective function is continuous and therefore an iterative gradient ascent algorithm can be defined to efficiently search the space. Moreover this results in a method which is less memory demanding and one that is very suitable to a parallel processing architecture. Experimental results shows that this approach is efficient for getting a robust initial guess to the 3D shape of an object in view.

Robust shape from depth images with GR2T

“…Note that this modelling links explicitly the observed quantity x from the cameras with the additive perturbation . The first two functions ðF 1 ; F 2 Þ link the pixel positions to the latent 3D locations and was used in Ruttle et al's modelling to infer shape-from-silhouettes [36,39] (cf. Eq.…”

“…Both Ruttle and Kim [36,39] considered that the camera matrices associated with each recorded image are known exactly. In practice however, despite careful calibration, errors occur on the extrinsic camera parameters.…”

“…In this paper, we propose first to extend the link function (11) to deal with data captured with depth cameras, such that shapefrom-silhouettes framework [36] becomes a shape-from-depth (SfD) approach (see Section 3). Second, we extend GR 2 T framework to estimate the extrinsic parameters of the cameras which are nuisance parameters in the context of 3D shape inference: the extrinsic camera parameters are not of interest but they need to be accurately estimated to allow for an accurate shape reconstruction.…”

“…Using a differentiable Gaussian kernel p allows them to optimise the cost function with gradient ascent algorithms that are suitable for parallel architecture [2]. The link function used by Ruttle et al [36] corresponds to the relation between the 3D position H ¼ ðh 1 ; h 2 ; h 3 Þ in the real world and the pixel position x ¼ ðx 1 ; x 2 Þ where H is projected using a projection matrix P associated with a pinhole camera model:…”

“…Ruttle et al [36] proposed a kernel density estimate as a cost function to infer shape from silhouettes as an extension to the classic 3D histogram for estimating the visual hull by cone intersections [37,38]. Using a differentiable Gaussian kernel p allows them to optimise the cost function with gradient ascent algorithms that are suitable for parallel architecture [2].…”