What Is Learned in Deep Uncalibrated Photometric Stereo?

Chen, Guanying; Waechter, Michael; Shi, Boxin; Wong, Kwan-Yee K.; Matsushita, Yasuyuki

doi:10.1007/978-3-030-58568-6_44

Cited by 38 publications

(83 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, we will extend our DR-PSN to the uncalibrated photometric stereo, which will benefit wider practical applications. We will design an illumination direction prediction network, which has already been investigated in some deep learning-based uncalibrated photometric stereo methods [11], [58], to estimate the lights from the input image, instead of inputting the calibrated illumination directions.…”

Section: Discussionmentioning

confidence: 99%

Recovering Surface Normal and Arbitrary Images: A Dual Regression Network for Photometric Stereo

Dong

Chen

2021

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

show abstract

Section: Discussionmentioning

confidence: 99%

Recovering Surface Normal and Arbitrary Images: A Dual Regression Network for Photometric Stereo

Dong

Chen

2021

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

show abstract

“…Table 2 shows that our method achieves competitive results with an average MAE normal of 9.15 • , having the second best performance overall. The best performing method [9] uses a four-stage cascade structure, making it complex and deep. On the contrary, our searched architecture is light and it can achieve such accuracy with 2.4M fewer parameters.…”

Section: Inferencementioning

confidence: 99%

“…However, these methods still rely on the other assumption of calibrated setting i.e., the light source directions are given at test time, limiting their practical application. Accordingly, uncalibrated deep PS methods that can provide results comparable to calibrated PS networks are becoming more and more popular [6,27,9].…”

Section: Introductionmentioning

confidence: 99%

“…The impressive results demonstrated by deep uncalibrated PS methods have a few critical issues: the network architecture is manually designed, and therefore, such networks are typically not optimally efficient and have a large memory footprint [27,6,8,9]. Moreover, the authors of such networks conduct many experiments to explore the effect of empirically selected operations and tune hyperparameters.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Neural Architecture Search for Efficient Uncalibrated Deep Photometric Stereo

Sarno¹,

Kumar²,

Kaya³

et al. 2022

2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

We present an automated machine learning approach for uncalibrated photometric stereo (PS). Our work aims at discovering lightweight and computationally efficient PS neural networks with excellent surface normal accuracy. Unlike previous uncalibrated deep PS networks, which are handcrafted and carefully tuned, we leverage differentiable neural architecture search (NAS) strategy to find uncalibrated PS architecture automatically. We begin by defining a discrete search space for a light calibration network and a normal estimation network, respectively. We then perform a continuous relaxation of this search space and present a gradient-based optimization strategy to find an efficient light calibration and normal estimation network. Directly applying the NAS methodology to uncalibrated PS is not straightforward as certain task-specific constraints must be satisfied, which we impose explicitly. Moreover, we search for and train the two networks separately to account for the Generalized Bas-Relief (GBR) ambiguity. Extensive experiments on the DiLiGenT dataset show that the automatically searched neural architectures performance compares favorably with the state-of-the-art uncalibrated PS methods while having a lower memory footprint.

show abstract

“…The function ρ(n(x), l s , v) gives the BRDF value, ζ a (n(x), l s ) = max(n(x) T l s , 0) accounts for the attached shadow, and ζ c (x) ∈ {0, 1} assign 0 or 1 value to x depending on whether it lies in the cast shadow region or not. e s ∈ R + is a scalar for light intensity value, and n(x) ∈ R 3×1 is the surface normal vector at point x. Eq:(1) is most-widely used photometric stereo formulation which generally works well in practise [9,30,28,65,13,11]. 1.…”

Section: Photometric Stereomentioning

confidence: 99%

Uncalibrated Neural Inverse Rendering for Photometric Stereo of General Surfaces

Kaya¹,

Kumar²,

Oliveira³

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

This paper presents an uncalibrated deep neural network framework for the photometric stereo problem. For training models to solve the problem, existing neural network-based methods either require exact light directions or groundtruth surface normals of the object or both. However, in practice, it is challenging to procure both of this information precisely, which restricts the broader adoption of photometric stereo algorithms for vision application. To bypass this difficulty, we propose an uncalibrated neural inverse rendering approach to this problem. Our method first estimates the light directions from the input images and then optimizes an image reconstruction loss to calculate the surface normals, bidirectional reflectance distribution function value, and depth. Additionally, our formulation explicitly models the concave and convex parts of a complex surface to consider the effects of interreflections in the image formation process. Extensive evaluation of the proposed method on the challenging subjects generally shows comparable or better results than the supervised and classical approaches.

show abstract

What Is Learned in Deep Uncalibrated Photometric Stereo?

Cited by 38 publications

References 45 publications

Recovering Surface Normal and Arbitrary Images: A Dual Regression Network for Photometric Stereo

Recovering Surface Normal and Arbitrary Images: A Dual Regression Network for Photometric Stereo

Neural Architecture Search for Efficient Uncalibrated Deep Photometric Stereo

Uncalibrated Neural Inverse Rendering for Photometric Stereo of General Surfaces

Contact Info

Product

Resources

About