Solving Uncalibrated Photometric Stereo Using Fewer Images by Jointly Optimizing Low-rank Matrix Completion and Integrability

Sengupta, Shamik; Zhou, Hao; Forkel, Walter; Goldstein, Tom; Jacobs, David W.

doi:10.1007/s10851-017-0772-y

Cited by 11 publications

(6 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…with Ω ⊂ R 2 the mask of the object to reconstruct, I i : Ω → R the i-th input graylevel image, ρ the reflectance (albedo) map, n the normal map (which encodes the 3Dgeometry), and s i ∈ R 3 a vector representing the incident lighting in the i-th image (in intensity and direction). Most of recent works on photometric stereo have focused on relaxing the assumptions of Lambertian reflectance (i.e., handling surfaces which exhibit a specular behavior) [4,21,11,29] and calibrated directional lighting (i.e., handling unknown or non-uniform lighting) [5,10,13,22], see for instance [23] for some discussion and [3] for a state-ofthe-art joint solution to both issues using deep neural networks. However, in all of these recent works the object to reconstruct is assumed to be segmented a priori: the whole pipeline relies on the knowledge of the domain Ω.…”

Section: Variational Methods For Photometric Stereo and Segmentationmentioning

confidence: 99%

Photometric Segmentation: Simultaneous Photometric Stereo and Masking

Haefner

Quéau

Cremers

2019

2019 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

This work is concerned with both the 3D-reconstruction of an object using photometric stereo, and its 2Dsegmentation from the background. In contrast with previous works on photometric stereo which assume that a mask of the area of interest has been computed beforehand, we formulate 3D-reconstruction and 2D-segmentation as a joint problem. The proposed variational solution combines a differential formulation of photometric stereo with the classic Chan-Vese model for active contours. Given a set of photometric stereo images, this solution simultaneously infers a binary mask of the object of interest and a depth map representing its 3D-shape. Experiments on real-world datasets confirm the soundness of simultaneously solving both these classic computer vision problems, as the joint approach considerably simplifies the overall 3D-scanning process for the end-user.

show abstract

Section: Variational Methods For Photometric Stereo and Segmentationmentioning

confidence: 99%

Photometric Segmentation: Simultaneous Photometric Stereo and Masking

Haefner

Quéau

Cremers

2019

2019 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

show abstract

“…Papadhimitri et al [52] presents a closed-form solution by detecting local diffuse reflectance maxima (LDR). Other methods assume perspective projection [51], specularities [21,18], low-rank [59], interreflections [9] or symmetry properties of BRDFs [64,73,44].…”

Section: Related Workmentioning

confidence: 99%

Uncalibrated Neural Inverse Rendering for Photometric Stereo of General Surfaces

Kaya¹,

Kumar²,

Oliveira³

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

This paper presents an uncalibrated deep neural network framework for the photometric stereo problem. For training models to solve the problem, existing neural network-based methods either require exact light directions or groundtruth surface normals of the object or both. However, in practice, it is challenging to procure both of this information precisely, which restricts the broader adoption of photometric stereo algorithms for vision application. To bypass this difficulty, we propose an uncalibrated neural inverse rendering approach to this problem. Our method first estimates the light directions from the input images and then optimizes an image reconstruction loss to calculate the surface normals, bidirectional reflectance distribution function value, and depth. Additionally, our formulation explicitly models the concave and convex parts of a complex surface to consider the effects of interreflections in the image formation process. Extensive evaluation of the proposed method on the challenging subjects generally shows comparable or better results than the supervised and classical approaches.

show abstract

“…[11,13]. Low-rank approximation was also considered for solving the uncalibrated case by jointly minimizing the rank and enforcing integrability [15]. Fig.…”

Section: Related Workmentioning

confidence: 99%

Low-Rank Registration of Images Captured Under Unknown, Varying Lighting

Pizenberg

Quéau

Elmoataz

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Photometric stereo infers the 3D-shape of a surface from a sequence of images captured under moving lighting and a static camera. However, in real-world scenarios the viewing angle may slightly vary, due to vibrations induced by the camera shutter, or when the camera is hand-held. In this paper, we put forward a low-rank affine registration technique for images captured under unknown, varying lighting. Optimization is carried out using convex relaxation and the alternating direction method of multipliers. The proposed method is shown to significantly improve 3D-reconstruction by photometric stereo on unaligned real-world data, and an open-source implementation is made available.

show abstract

Solving Uncalibrated Photometric Stereo Using Fewer Images by Jointly Optimizing Low-rank Matrix Completion and Integrability

Cited by 11 publications

References 37 publications

Photometric Segmentation: Simultaneous Photometric Stereo and Masking

Photometric Segmentation: Simultaneous Photometric Stereo and Masking

Uncalibrated Neural Inverse Rendering for Photometric Stereo of General Surfaces

Low-Rank Registration of Images Captured Under Unknown, Varying Lighting

Contact Info

Product

Resources

About