Towards Learning Neural Representations from Shadows

Tiwary, Kushagra; Klinghoffer, Tzofi; Raskar, Ramesh

doi:10.48550/arxiv.2203.15946

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Shadow gradients In single view optimization [60], shape from shadows [57], or direct optimization of the position/direction of analytical light sources, shadow ray visibility gradients [36,3] are highly beneficial. However, in our multi-view setting (50+ views), similar to Loubet et al [36], we observed that gradients of diffuse scattering are negligible compared to the gradients of primary visibility.…”

Section: Direct Illuminationmentioning

confidence: 99%

Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising

Hasselgren¹,

Hofmann²,

Munkberg³

2022

Preprint

View full text Add to dashboard Cite

Recent advances in differentiable rendering have enabled high-quality reconstruction of 3D scenes from multi-view images. Most methods rely on simple rendering algorithms: pre-filtered direct lighting or learned representations of irradiance. We show that a more realistic shading model, incorporating ray tracing and Monte Carlo integration, substantially improves decomposition into shape, materials & lighting. Unfortunately, Monte Carlo integration provides estimates with significant noise, even at large sample counts, which makes gradient-based inverse rendering very challenging. To address this, we incorporate multiple importance sampling and denoising in a novel inverse rendering pipeline. This substantially improves convergence and enables gradient-based optimization at low sample counts. We present an efficient method to jointly reconstruct geometry (explicit triangle meshes), materials, and lighting, which substantially improves material and light separation compared to previous work. We argue that denoising can become an integral part of high quality inverse rendering pipelines.Preprint. Under review.

show abstract

Section: Direct Illuminationmentioning

confidence: 99%

Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising

Hasselgren¹,

Hofmann²,

Munkberg³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Different from shape-from-shadow methods [21,55], our method does not require direct supervision for shadow rendering. We rely on image reconstruction loss for optimization.…”

Section: Optimizationmentioning

confidence: 99%

S$^3$-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpoint

Yang¹,

Chen²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we address the "dual problem" of multi-view scene reconstruction in which we utilize single-view images captured under different point lights to learn a neural scene representation. Different from existing single-view methods which can only recover a 2.5D scene representation (i.e., a normal / depth map for the visible surface), our method learns a neural reflectance field to represent the 3D geometry and BRDFs of a scene. Instead of relying on multi-view photoconsistency, our method exploits two information-rich monocular cues, namely shading and shadow, to infer scene geometry. Experiments on multiple challenging datasets show that our method is capable of recovering 3D geometry, including both visible and invisible parts, of a scene from single-view images. Thanks to the neural reflectance field representation, our method is robust to depth discontinuities. It supports applications like novel-view synthesis and relighting. Our code and model can be found at https://ywq.github.io/s3nerf.

show abstract

Towards Learning Neural Representations from Shadows

Cited by 2 publications

References 0 publications

Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising

Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising

S$^3$-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpoint

Contact Info

Product

Resources

About