Towards a complete dense geometric and photometric reconstruction under varying pose and illumination

Abstract

This paper proposes a novel framework to construct a geometric and photometric model of a viewed object that can be used for visualisation in arbitrary pose and illumination. The method is solely based on images and does not require any specialised equipment. We assume that the object has a piece-wise smooth surface and that its reflectance can be modelled using a parametric bidirectional reflectance distribution function. Without assuming any prior knowledge on the object, geometry and reflectance have to be estimated simultaneously and occlusion and shadows have to be treated consistently. We exploit the geometric and photometric consistency using the fact that surface orientation and reflectance are local invariants. In a first implementation, we demonstrate the method using a Lambertian object placed on a turn-table and illuminated by a number of unknown point light-sources. A discrete voxel model is initialised to the visual hull and voxels identified as inconsistent with the invariants are removed iteratively. The resulting model is used to render images in novel pose and illumination.

A Differential Volumetric Approach to Multi-View Photometric Stereo

“…Reconstructing accurate 3D geometry of a volume has been a challenging area in computer vision. Most of the research trying to solve this problem has been developed by merging multi-view methods for coarse reconstruction [10], with techniques based on shading information for providing high frequency details of the surface [30,46,48,31,5,8] rather than based on a topological evolution of the surface [16]. However, regarding the refinement, several methods take inspiration from Shape from Shading [14] to extract 3D geometry from a single image (MVSfS) and consider shape refinement resulting from single shading cues [50,49,3].…”

A Survey of Photometric Stereo Techniques

“…This has the advantage of providing readily available occlusion information while the surface is deformed by the optimization. Weber et al [2002] use a voxel representation and consider objects on a turntable. They carve away voxels for which the predictions of the Lambertian model-given light source positions-disagree with the recorded intensities even for the best fitting normal.…”

Consensus Multi-View Photometric Stereo

“…Several methods try to circumvent the problem of unknown occlusions or visibility with a proxy object that is refined iteratively. Weber et al [17] apply a voxel-based approach. It relies on object silhouettes and iteratively carves away voxels outside the consistency hull.…”