A Variational Approach to Shape-from-Shading Under Natural Illumination

Abstract

A numerical solution to shape-from-shading under natural illumination is presented. It builds upon an augmented Lagrangian approach for solving a generic PDE-based shape-from-shading model which handles directional or spherical harmonic lighting, orthographic or perspective projection, and greylevel or multi-channel images. Real-world applications to shading-aware depth map denoising, refinement and completion are presented.

Photometric Depth Super-Resolution

“…Obviously, even more ambiguities arise under more realistic lighting and reflectance assumptions: any image can be explained by a flat shape illuminated uniformly but painted in a complex manner, by a white and frontally-lit surface with a complex geometry, or by a white planar surface illuminated in a complex manner [26]. Shape-from-shading under uniform reflectance but natural lighting has been studied [27], [28], [29], [30], but the case with unknown reflectance requires the introduction of additional priors [31]. This can be avoided by actively controlling the lighting, a variant of shape-from-shading known as photometric stereo which allows to estimate both shape and reflectance [32].…”

“…All three problems (29), (30) and (31) are linear least-squares problems which we solve using the conjugate gradient method on the normal equations.…”

“…24: Impact of the number of images n on the accuracy of the albedo and depth estimates using our multi-shot photometric stereo approach (γ = 0.01 in this experiment). The range n ∈ [10,30] represents a reasonable compromise between accuracy and runtime. Figure 26 shows four qualitative comparisons against the state-of-the-art, on real-world data from Figures 1 and 6 in the main paper, which was captured with an Asus Xtion Pro Live camera (scaling factor of 4).…”

Fight Ill-Posedness with Ill-Posedness: Single-shot Variational Depth Super-Resolution from Shading

“…We now describe an algorithm for effectively solving the variational problem (18), which is both nonsmooth and nonconvex. In order to tackle the nonlinear dependency upon the depth and its gradient arising from shape-from-shading and minimal surface regularisation, we follow [47] and introduce an auxiliary variable θ := (z, ∇z), then rewrite (18) as a constrained optimisation problem:…”

Numerical shape-from-shading revisited

“…While most of the methods assume a point light source, Queau et al [29] proposed a shape-from-shading method under natural illumination. They used a variational method for ensuring smoothness of surface normal through regularization by solving partial differential equations.…”