Gradient and curvature from the photometric-stereo method, including local confidence estimation

Abstract

The photometric-stereo method is one technique for three-dimensional shape determination that has been implemented in a variety of experimental settings and that has produced consistently good results. The idea is to use intensity values recorded from multiple images obtained from the same viewpoint but under different conditions of illumination. The resulting radiometric constraint makes it possible to obtain local estimates of both surface orientation and surface curvature without requiring either global smoothness assumptions or prior image segmentation. Photometric stereo is moved one step closer to practical possibility by a description of an experimental setting in which surface gradient estimation is achieved on full-frame video data at nearvideo-frame rates (i.e., 15 Hz). The implementation uses commercially available hardware. Reflectance is modeled empirically with measurements obtained from a calibration sphere. Estimation of the gradient (p, q) requires only simple table lookup. Curvature estimation additionally uses the reflectance map R(p, q). The required lookup table and reflectance maps are derived during calibration. Because reflectance is modeled empirically, no prior physical model of the reflectance characteristics of the objects to be analyzed is assumed. At the same time, if a good physical model is available, it can be retrofitted to the method for implementation purposes. Photometric stereo is subject to error in the presence of cast shadows and interreflection. No purely local technique can succeed because these phenomena are inherently nonlocal. Nevertheless, it is demonstrated that one can exploit the redundancy in three-light-source photometric stereo to detect locally, in most cases, the presence of cast shadows and interreflection. Detection is facilitated by the explicit inclusion of a local confidence estimate in the lookup table used for gradient estimation.

Towards a complete dense geometric and photometric reconstruction under varying pose and illumination

“…As a consequence, the reconstruction requires a prior on shape [7,1] and regularisation [16]. This problem can be reduced by using several light-sources as in photometric stereo [15] but the technique is limited by the restriction to one view. Our method is related to shape from shading techniques in that we are exploiting the dense irradiance values for geometric constraints on the surface orientation.…”

Lump detection with a gelsight sensor

“…When the pad is pressed against a rigid surface, the membrane takes on the surface's 3D topography, allowing the camera to record a shaded image indicative of that topography (see Figure 2). Through the use of photometric stereo [14], [1], the sensor can produce a detailed image of the membrane's 3D shape. As shown in Figure 2a and 2b, an Oreo cookie is pressed against the GelSight sensor.…”

Dense Photometric Stereo: A Markov Random Field Approach

“…While approaches in photometric stereo using two views with known albedos [44], three views [15], four views [7], [35], [3], more views [22], complex reflectance models [28], [37], [18], [35], lookup tables [44], [45], reference objects [16], [13], [10], and novel object representation [4] have been reported, photometric stereo is still considered to be a difficult problem in the presence of shadows and specular highlights, and for objects with complex material and geometry.…”