Neural Reflectance for Shape Recovery with Shadow Handling

Li, Junxuan; Li, Hongdong

doi:10.1109/cvpr52688.2022.01574

Cited by 30 publications

(43 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…to deal with depth discontinuities [25]. It is desirable to obtain a more complete scene reconstruction (including both visible and invisible parts) from single-view images.…”

Section: (B)) Besides Methods Relying On Surface Normal Representatio...mentioning

confidence: 99%

“…Following existing nearfield photometric stereo methods [36,46], we assume a calibrated perspective camera and known point light positions. Instead of representing the visible surface with a normal / depth map like others [25,36,46], we adopt a 3D neural field representation [3,37,41] to describe the 3D scene.…”

Section: Methodsmentioning

confidence: 99%

“…For near-field PS problem, methods based on PDE [42] and deep learning [29,32,36,46] have been proposed. More recently, Li et al [25] proposes a coordinate-based MLP to represent the normal map of the visible surface assuming directional lights. In contrast, our method represents a scene with a continuous volume and recovers the full 3D scene geometry under a near-field setup.…”

Section: Related Workmentioning

confidence: 99%

“…Following [16,25], we model the isotropic specular reflectance by a weighted combination of Sphere Gaussian (SG) bases, which demonstrates better results in modeling specular effects than the parametric Microfacet model [20]. The specular component ρ s is hence written as…”

Section: Physics-based Color Renderingmentioning

confidence: 99%

See 3 more Smart Citations

S$^3$-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpoint

Yang¹,

Chen²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we address the "dual problem" of multi-view scene reconstruction in which we utilize single-view images captured under different point lights to learn a neural scene representation. Different from existing single-view methods which can only recover a 2.5D scene representation (i.e., a normal / depth map for the visible surface), our method learns a neural reflectance field to represent the 3D geometry and BRDFs of a scene. Instead of relying on multi-view photoconsistency, our method exploits two information-rich monocular cues, namely shading and shadow, to infer scene geometry. Experiments on multiple challenging datasets show that our method is capable of recovering 3D geometry, including both visible and invisible parts, of a scene from single-view images. Thanks to the neural reflectance field representation, our method is robust to depth discontinuities. It supports applications like novel-view synthesis and relighting. Our code and model can be found at https://ywq.github.io/s3nerf.

show abstract

“…to deal with depth discontinuities [25]. It is desirable to obtain a more complete scene reconstruction (including both visible and invisible parts) from single-view images.…”

Section: (B)) Besides Methods Relying On Surface Normal Representatio...mentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Physics-based Color Renderingmentioning

confidence: 99%

See 2 more Smart Citations

S$^3$-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpoint

Yang¹,

Chen²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Nonetheless, such a model restricts methods' performance on objects with more general (e.g., anisotropic) materials, while modeling general reflectance is challenging due to extra unknowns, which eventually make UPS intractable. Other works (e.g., [25,56,57]) notice the benefits of the shadow cues in utilizing global shape-light information to solve PS/UPS because the shadow reflects the interaction of shape and light [24,59]. However, these methods either fail to exploit the shadow cues due to the lack of a differentiable path from the shadow to the concerned unknowns like shape [25], or the shadow cues have limited effects on the visible shape reconstruction due to the implicit shape representation [56,57].…”

Section: Introductionmentioning

confidence: 99%

DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering

Li¹,

Zheng²,

Shi³

et al. 2023

Preprint

View full text Add to dashboard Cite

Uncalibrated photometric stereo (UPS) is challenging due to the inherent ambiguity brought by the unknown light. Although the ambiguity is alleviated on non-Lambertian objects, the problem is still difficult to solve for more general objects with complex shapes introducing irregular shadows and general materials with complex reflectance like anisotropic reflectance. To exploit cues from shadow and reflectance to solve UPS and improve performance on general materials, we propose DANI-Net, an inverse rendering framework with differentiable shadow handling and anisotropic reflectance modeling. Unlike most previous methods that use non-differentiable shadow maps and assume isotropic material, our network benefits from cues of shadow and anisotropic reflectance through two differentiable paths. Experiments on multiple real-world datasets demonstrate our superior and robust performance.

show abstract

PS-NeRF: Neural Inverse Rendering for Multi-view Photometric Stereo

Yang

Chen

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Neural Reflectance for Shape Recovery with Shadow Handling

Cited by 30 publications

References 20 publications

S$^3$-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpoint

S$^3$-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpoint

DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering

PS-NeRF: Neural Inverse Rendering for Multi-view Photometric Stereo

Contact Info

Product

Resources

About