Optimisation of Classic Photometric Stereo by Non-convex Variational Minimisation

Abstract

Estimating shape and appearance of a three dimensional object from a given set of images is a classic research topic that is still actively pursued. Among the various techniques available, photometric stereo is distinguished by the assumption that the underlying input images are taken from the same point of view but under different lighting conditions. The most common techniques provide the shape information in terms of surface normals. In this work, we instead propose to minimise a much more natural objective function, namely the reprojection error in terms of depth. Minimising the resulting non-trivial variational model for photometric stereo allows to recover the depth of the photographed scene directly. As a solving strategy, we follow an approach based on a recently published optimisation scheme for non-convex and non-smooth cost functions.The main contributions of our paper are of theoretical nature. A technical novelty in our framework is the usage of matrix differential calculus. We supplement our approach by a detailed convergence analysis of the resulting optimisation algorithm and discuss possibilities to ease the computational complexity. At hand of an experimental evaluation we discuss important properties of the method. Overall, our strategy achieves more accurate results than competing approaches. The experiments also highlights some practical aspects of the underlying optimisation algorithm that may be of interest in a more general context. G. Radow · L. Hoeltgen · M. Breuß Chair for Applied Mathematics

On the well-posedness of uncalibrated photometric stereo under general lighting

“…Such a simple least-squares approach may be replaced by robust variational or learning-based strategies to ensure robustness [13,17,33,34]. There also exist numerical solutions for handling non-Lambertian reflectance models [7,15,21,40], non-distant light sources [18,22,29,32], or the ill-posed cases where m = 2 [16,20,24,31] or m = 1 [5,8,39,45].…”

“…where i o (x) ∈ R 11 is the "orthographic integrability vector" containing factors c i, j u (x) and c i, j v (x), and a ∈ R 11 contain the minors A i, j k,l of A appearing in (34). Let us assume that there exist at least 11 points x ∈ Ω such that the matrix formed by concatenating row-wise the vectors i o (x) is full-rank (in practice, this property is satisfied as long as the surface is not trivial).…”

Photometric Stereo with Non-Lambertian Preprocessing and Hayakawa Lighting Estimation for Highly Detailed Shape Reconstruction

“…Despite its long history in computer vision [49,50], PS is still a fundamentally challenging research problem due to the unknown reflectance and global lighting effects of real-world objects [41]. For the sake of simplicity, the lighting positions and directions are often assumed to be known across the PS research community [4,28,38,48]. However, the real world applications of PS mostly deal with data represented by images acquired under unknown light conditions; see [5], [11], and [21].…”

Ascertaining the Ideality of Photometric Stereo Datasets under Unknown Lighting

“…Though in the research community, the position of the light sources is generally assumed to be known [9][10][11], real-world applications of PS often deal with datasets acquired under unknown light conditions [12,13]. In [14], Hayakawa has shown that the lighting directions can be identified directly from the data when at least six images with different illumination are available (see [14] and Section 2 for a proof), yielding the possibility to apply PS to field measurement scenarios, as in the case of archaeological excavations [15].…”