Intrinsic Textures for Relightable Free-Viewpoint Video

Imber, J.; Guillemaut, Jean-Yves; Hilton, Adrian

doi:10.1007/978-3-319-10605-2_26

Cited by 9 publications

(6 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We try to minimize the color difference of different data sets by using uniform lighting when capturing the data set images. In future work, multisource co-color-correction methods can be used to adjust the colors of different database images to be consistent, and intrinsic image decomposition methods can be adopted together with relighting techniques to increase the realism of the generated video [40].…”

Section: Discussionmentioning

confidence: 99%

SPA: Sparse Photorealistic Animation Using a Single RGB-D Camera

Yang

Liu

et al. 2016

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

Section: Discussionmentioning

confidence: 99%

SPA: Sparse Photorealistic Animation Using a Single RGB-D Camera

Yang

Liu

et al. 2016

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

“…Subsequent work has focused on reducing storage requirements [Furukawa et al 2002], accelerating relighting [Malzbender et al 2001], reducing the number of required photographs , or acquisition under uncontrolled lighting either by limiting to specialized scenes (e.g., human subjects) [Guo et al 2019;Li et al 2013], landmarks [Haber et al 2009], etc. ), specialized lighting (e.g., outdoor natural lighting [Hauagge et al 2014]), or simplified transport (e.g., lambertian reflectance [Imber et al 2014]). Machine learning methods have been used to further reduce the number of required images for single view relighting [Meka et al 2019;Ren et al 2015;Sun et al 2019;] and very recently for multi-view relighting [Chen et al 2020;Kanamori and Endo 2018;Meshry et al 2019;].…”

Section: Image-based Solutionsmentioning

confidence: 99%

Deferred neural lighting

et al. 2020

View full text Add to dashboard Cite

We present deferred neural lighting, a novel method for free-viewpoint relighting from unstructured photographs of a scene captured with handheld devices. Our method leverages a scene-dependent neural rendering network for relighting a rough geometric proxy with learnable neural textures. Key to making the rendering network lighting aware are radiance cues: global illumination renderings of a rough proxy geometry of the scene for a small set of basis materials and lit by the target lighting. As such, the light transport through the scene is never explicitely modeled, but resolved at rendering time by a neural rendering network. We demonstrate that the neural textures and neural renderer can be trained end-to-end from unstructured photographs captured with a double hand-held camera setup that concurrently captures the scene while being lit by only one of the cameras' flash lights. In addition, we propose a novel augmentation refinement strategy that exploits the linearity of light transport to extend the relighting capabilities of the neural rendering network to support other lighting types (e.g., environment lighting) beyond the lighting used during acquisition (i.e., flash lighting). We demonstrate our deferred neural lighting solution on a variety of real-world and synthetic scenes exhibiting a wide range of material properties, light transport effects, and geometrical complexity.

show abstract

“…Our technique is related to Precomputed Radiance Transfer techniques (PRT) (Sloan et al, 2002;Ng et al, 2003;Sun et al, 2007;Imber et al, 2014), in that we factorize rendering and represent the estimated data in efficient bases.…”

Section: Related Workmentioning

confidence: 99%

Interactive Relighting of Virtual Objects under Environment Lighting

Michiels¹,

Put²,

Bekaert³

2015

Proceedings of the 10th International Conference on Computer Graphics Theory and Applications

View full text Add to dashboard Cite

Abstract:Current relighting applications often constrain one or several factors of the rendering equation to keep the rendering speed real-time. For example, visibility is often precalculated and animations are not allowed, changes in lighting are limited to simple rotation or the lighting is not very detailed. Other techniques compromise on quality and often coarsely tabulate BRDF functions. In order to solve these problems, some techniques have started to use spherical radial basis functions. However, solving the triple product integral does not guarantee interactivity. In order to dynamically change lighting conditions or alter scene geometry and materials, these three factors need to be converted to the SRBF representation in a fast manner. This paper presents a method to perform the SRBF data construction and rendering in real-time. To support dynamic high-frequency lighting, a multiscale residual transformation algorithm is applied. Area lights are detected through a peak detection algorithm. By using voxel cone tracing and a subsampling scheme, animated geometry casts soft shadows dynamically. We demonstrate the effectiveness of our method with a real-time application. Users can shine with multiple light sources onto a camera and the animated virtual scene is relit accordingly.

show abstract

Intrinsic Textures for Relightable Free-Viewpoint Video

Cited by 9 publications

References 30 publications

SPA: Sparse Photorealistic Animation Using a Single RGB-D Camera

SPA: Sparse Photorealistic Animation Using a Single RGB-D Camera

Deferred neural lighting

Interactive Relighting of Virtual Objects under Environment Lighting

Contact Info

Product

Resources

About