Quantized Point‐Based Global Illumination

Self Cite

Acquired 3D point clouds make possible quick modeling of virtual scenes from the real world. With modern 3D capture pipelines, each point sample often comes with additional attributes such as normal vector and color response. Although rendering and processing such data has been extensively studied, little attention has been devoted using the light transport hidden in the recorded per‐sample color response to relight virtual objects in visual effects (VFX) look‐dev or augmented reality (AR) scenarios. Typically, standard relighting environment exploits global environment maps together with a collection of local light probes to reflect the light mood of the real scene on the virtual object. We propose instead a unified spatial approximation of the radiance and visibility relationships present in the scene, in the form of a colored point cloud. To do so, our method relies on two core components: High Dynamic Range (HDR) expansion and real‐time Point‐Based Global Illumination (PBGI). First, since an acquired color point cloud typically comes in Low Dynamic Range (LDR) format, we boost it using a single HDR photo exemplar of the captured scene that can cover part of it. We perform this expansion efficiently by first expanding the dynamic range of a set of renderings of the point cloud and then projecting these renderings on the original cloud. At this stage, we propagate the expansion to the regions not covered by the renderings or with low‐quality dynamic range by solving a Poisson system. Then, at rendering time, we use the resulting HDR point cloud to relight virtual objects, providing a diffuse model of the indirect illumination propagated by the environment. To do so, we design a PBGI algorithm that exploits the GPU's geometry shader stage as well as a new mipmapping operator, tailored for G‐buffers, to achieve real‐time performances. As a result, our method can effectively relight virtual objects exhibiting diffuse and glossy physically‐based materials in real time. Furthermore, it accounts for the spatial embedding of the object within the 3D environment. We evaluate our approach on manufactured scenes to assess the error introduced at every step from the perfect ground truth. We also report experiments with real captured data, covering a range of capture technologies, from active scanning to multiview stereo reconstruction.

Section: Point Based Global Illuminationmentioning

confidence: 99%

High Dynamic Range Point Clouds for Real‐Time Relighting

Sabbadin

Palma

Banterle

et al. 2019

Self Cite

“…A number of approaches have been proposed to improve the cut computation, including importance‐driven point projection based on an initial clustering [MW11], cut picking algorithm for HDR imaging [Tab12], and tree‐cut/microbuffers factorization based on spatial coherence [WHB*13]. The PBGI memory issue has been tackled with an out‐of‐core framework for PBGI, providing a cache‐coherent tree construction and traversal [KTO11], and with an in‐core solution which quantizes all tree nodes against a small set of representatives, learned on‐the‐fly [BB12].…”

Section: Previous Workmentioning

confidence: 99%

“…At caching time, we can retain only G 0 (the root average coefficient vector), discard the other average vectors and store only the detail vectors in the nodes, ignoring the ones with a L 2 norm smaller than a user‐defined compression threshold (set to 0.002 in our experiments). To avoid storing the entire list of nodes vectors at any intermediate state, we compute this compressed representation during a post‐order depth‐first traversal of the PBGI tree [BB12]. At rendering time, we reconstruct the radiance coefficient vector of a given node j as:

…”

Section: Wavelet Pbgimentioning

confidence: 99%

Wavelet Point‐Based Global Illumination

Wang

Meng

Boubekeur

2015

Self Cite

b) WPBGI (c) BPT (a) PBGI Figure 1: Compared to traditional PBGI (a), our approach (b) models accurately non-diffuse indirect lighting effects and appears as an efficient substitute to bidirectional path tracing (c), trading a moderate image degradation for up to a 10x speed-up in our experiments. AbstractPoint-Based Global Illumination (PBGI) is a popular rendering method in special effects and motion picture productions. This algorithm provides a diffuse global illumination solution by caching radiance in a mesh-less hierarchical data structure during a pre-process, while solving for visibility over this cache, at rendering time and for each receiver, using microbuffers, which are localized depth and color buffers inspired from real time rendering environments. As a result, noise free ambient occlusion, indirect soft shadows and color bleeding effects are computed efficiently for high resolution image output and in a temporally coherent fashion. We propose an evolution of this method to address the case of non-diffuse inter-reflections and refractions. While the original PBGI algorithm models radiance using spherical harmonics, we propose to use wavelets parameterized on the direction space to better localize the radiance representation in the presence of highly directional reflectance. We also propose a new importance-driven adaptive microbuffer model to capture accurately incoming radiance at a point. Furthermore, we evaluate outgoing radiance using a fast wavelet radiance product and contain the induced larger memory footprint by encoding hierarchically the wavelets in the PBGI tree. As a result, our algorithm can handle non-lambertian BSDF in the light transport simulation, reproducing caustics and multiple reflections/refractions bounces with a similar quality to bidirectional path tracing in a large number of cases and for only a fraction of its computation time. Our approach is simple to implement and easy to integrate into any existing PBGI framework, with an intuitive control on the approximation error. We evaluate it on a collection of example scenes.

“…Buchholz and Boubekeur [BB12] proposed an in-core solution to this problem, learning a reduced set of node data vectors in high dimension and quantizing all tree nodes against the resulting look-up table.…”

Section: Previousmentioning

confidence: 99%

“…The PBGI accuracy entirely depends on the density of the initial sampling and the related memory issue has been tackled by Kontkanen et al [KTO11], who proposed an out‐of‐core framework for PBGI with cache‐coherent tree construction and traversal. Buchholz and Boubekeur [BB12] proposed an in‐core solution to this problem, learning a reduced set of node data vectors in high dimension and quantizing all tree nodes against the resulting look‐up table.…”

Section: Previous Workmentioning

confidence: 99%

Factorized Point Based Global Illumination

Wang

Huang

Buchholz

et al. 2013

Self Cite

The Point-Based Global Illumination (PBGI) algorithm is composed of two major steps: a caching step and a multiview rasterization step. At caching time, a dense point-sampling of the scene is shaded and organized in a spatial hierarchy, with internal nodes approximating the radiance of their subtrees using spherical harmonics. At rasterization time, a microbuffer is instantiated at the unprojected position of each image pixel (receiver). Then, a view-adaptive level-of-detail of the scene is extracted in the form of a tree cut and rasterized in the receiver's microbuffer, solving for visibility using a local variant of the z-buffer. Finally, the pixel color is computed by convolving its filled microbuffer with the surface BRDF. This noise-free indirect lighting method is widely used in the industry and captures several critical lighting effects, including ambient occlusion, color bleeding, (indirect) soft-shadows and environment lighting. However, we observe a large redundancy in this algorithm, both in cuts and receivers'microbuffers, which stems from their relatively low resolution. In this paper, we propose an evolution of PBGI which exploits spatial coherence to reduce these redundant computations. Starting from a similarity-based variational clustering of the receivers, we compute a single tree cut and rasterize a single microbuffer for each cluster. This per-cluster microbuffer provides a faithful approximation of the incident radiance for distant nodes and is composited over a receiver-specific microbuffer rasterizing only the closest nodes of the cluster's cut. This factorized approach is easy to integrate in any existing PBGI implementation and offers a significant rendering speed-up for a negligible and controllable approximation error.