Gradient-domain volumetric photon density estimation

Gruson, Adrien; Hua, Binh-Son; Vibert, Nicolas; Nowrouzezahrai, Derek; Hachisuka, Toshiya

doi:10.1145/3197517.3201363

Cited by 19 publications

(12 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our formulation allows us to pick non-consecutive distance dimensions to obtain complementary photon planes, or angular/area dimensions to obtain novel photon surfaces, all of which can be combined with MIS to robustly mitigate the singularities present in any one estimator alone. Our estimators are consistent with the density estimation framework and can be combined with orthogonal approaches such as gradient domain rendering [Lehtinen et al 2013;Kettunen et al 2015], as has previously been shown for beam/plane estimators [Gruson et al 2018].…”

Section: Related Worksupporting

confidence: 81%

Photon surfaces for robust, unbiased volumetric density estimation

et al. 2019

View full text Add to dashboard Cite

We generalize photon planes to photon surfaces : a new family of unbiased volumetric density estimators which we combine using multiple importance sampling. To derive our new estimators, we start with the extended path integral which duplicates the vertex at the end of the camera and photon subpaths and couples them using a blurring kernel. To make our formulation unbiased, however, we use a delta kernel to couple these two end points. Unfortunately, sampling the resulting singular integral using Monte Carlo is impossible since the probability of generating a contributing light path by independently sampling the two subpaths is zero. Our key insight is that we can eliminate the delta kernel and make Monte Carlo estimation practical by integrating any three dimensions analytically, and integrating only the remaining dimensions using Monte Carlo. We demonstrate the practicality of this approach by instantiating a collection of estimators which analytically integrate the distance along the camera ray and two arbitrary sampling dimensions along the photon subpath (e.g., distance, direction, surface area). This generalizes photon planes to curved "photon surfaces", including new "photon cone", "photon cylinder", "photon sphere", and multiple new "photon plane" estimators. These estimators allow us to handle light paths not supported by photon planes, including single scattering, and surface-to-media transport. More importantly, since our estimators have complementary strengths due to analytically integrating different dimensions of the path integral, we can combine them using multiple importance sampling. This combination mitigates singularities present in individual estimators, substantially reducing variance while remaining fully unbiased. We demonstrate our improved estimators on a number of scenes containing homogeneous media with highly anisotropic phase functions, accelerating both multiple scattering and single scattering compared to prior techniques.

show abstract

Section: Related Worksupporting

confidence: 81%

Photon surfaces for robust, unbiased volumetric density estimation

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Our proposed method can also be utilized for other gradient‐domain volume rendering techniques, like photon beams (gradient‐domain beam‐beam 3D density estimation [GHV∗18]). On top of our proposed method, we only did one small change, setting the photon density features to zero, as there are no photons in photon beams method.…”

Section: Resultsmentioning

confidence: 99%

“…Then, the gradient‐domain rendering is extended to other rendering methods, such as bidirectional path tracing by Manzi et al [MKA∗15], VCM by Sun et al [SSC∗17] and path reusing by Bauszat et al [BPE17]. Recently, Hua et al [HGNH17] proposed a gradient‐domain photon density estimation method, and then Gruson et al [GHV∗18] extended it to volumetric photon density estimation.…”

Section: Related Workmentioning

confidence: 99%

“…Gradient‐domain based rendering methods [KMA∗15] [LKL∗13] [MKA∗15] [SSC∗17] can improve the convergence of Monte Carlo renderings, by estimating pixel intensity along with its finite differences and performing a screened Poisson reconstruction. A recent work [GHV∗18] proposed a gradient‐domain methods for volumetric light transport in homogenous participating media.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised Image Reconstruction for Gradient‐Domain Volumetric Rendering

Xu¹,

Sun

Wang³

et al. 2020

Computer Graphics Forum

View full text Add to dashboard Cite

Gradient-domain rendering can highly improve the convergence of light transport simulation using the smoothness in image space. These methods generate image gradients and solve an image reconstruction problem with rendered image and the gradient images. Recently, a previous work proposed a gradient-domain volumetric photon density estimation for homogeneous participating media. However, the image reconstruction relies on traditional L1 reconstruction, which leads to obvious artifacts when only a few rendering passes are performed. Deep learning based reconstruction methods have been exploited for surface rendering, but they are not suitable for volume density estimation. In this paper, we propose an unsupervised neural network for image reconstruction of gradient-domain volumetric photon density estimation, more specifically for volumetric photon mapping, using a variant of GradNet with an encoded shift connection and a separated auxiliary feature branch, which includes volume based auxiliary features such as transmittance and photon density. Our network smooths the images on global scale and preserves the high frequency details on a small scale. We demonstrate that our network produces a higher quality result, compared to previous work. Although we only considered volumetric photon mapping, it's straightforward to extend our method for other forms, like beam radiance estimation. CCS Concepts • Computing methodologies → Neural network; Ray tracing; † Joint first author.

show abstract

“…adapt Path Tracing to Gradient-Domain Rendering. Subsequently, many other rendering methods have been adapted for the gradient domain: Gradient-Domain Bidirectional Path Tracing [Veach and Guibas 1995], Gradient-Domain [Bauszat et al 2017] Path Reusing [Bekaert et al 2002], Gradient-Domain [Hua et al 2017] Stochastic Progressive Photon Mapping [Hachisuka and Jensen 2009], Gradient-Domain [Sun et al 2017] Vertex Connection and Merging [Georgiev et al 2012], and rendering of homogenous volumes with Gradient-Domain adaptations [Gruson et al 2018] of Beam Radiance Estimates [Jarosz et al 2008], Progressive Photon Beams [Jarosz et al 2011] and Photon Planes [Bitterli and Jarosz 2017]. To enforce and exploit temporal coherence, Manzi et al [2016a] extend gradient computation to the time domain by mapping random seeds of light paths between successive animation frames, and solve a temporal extension of the screened Poisson equation.…”

Section: Denoisingmentioning

confidence: 99%

Deep convolutional reconstruction for gradient-domain rendering

2019

View full text Add to dashboard Cite

Input (Ours) NFOR [Bitterli et al. 2016] KPCN [Bako et al. 2017] NGPT (Ours) Ground Truth Fig. 1. Comparison of the primal-domain denoisers NFOR [Bitterli et al. 2016] and KPCN [Bako et al. 2017] to our gradient-domain reconstruction NGPT from very noisy equal-time inputs (8 samples for ours and 20 for others). Generally outperforming the comparison methods, our results show that gradient sampling is useful also in the context of non-linear neural image reconstruction, often resolving e.g. shadows better than techniques that do not make use of gradients. It has been shown that rendering in the gradient domain, i.e., estimating finite difference gradients of image intensity using correlated samples, and combining them with direct estimates of pixel intensities by solving a screened Poisson problem, often offers fundamental benefits over merely sampling pixel intensities. The reasons can be traced to the frequency content of the light transport integrand and its interplay with the gradient operator. However, while they often yield state of the art performance among algorithms that are based on Monte Carlo sampling alone, gradient-domain rendering algorithms have, until now, not generally been competitive with techniques that combine Monte Carlo sampling with post-hoc noise removal using sophisticated non-linear filtering. Drawing on the power of modern convolutional neural networks, we propose a novel reconstruction method for gradient-domain rendering. Our technique replaces the screened Poisson solver of previous gradient-domain techniques with a novel dense variant of the U-Net autoencoder, additionally taking auxiliary feature buffers as inputs. We optimize our network to minimize a perceptual image distance metric calibrated to the human visual system. Our results significantly improve the quality obtained from gradientdomain path tracing, allowing it to overtake state-of-the-art comparison techniques that denoise traditional Monte Carlo samplings. In particular, we observe that the correlated gradient samples-that offer information about the smoothness of the integrand unavailable in standard Monte Carlo sampling-notably improve image quality compared to an equally powerful neural model that does not make use of gradient samples.

show abstract

Gradient-domain volumetric photon density estimation

Cited by 19 publications

References 31 publications

Photon surfaces for robust, unbiased volumetric density estimation

Photon surfaces for robust, unbiased volumetric density estimation

Unsupervised Image Reconstruction for Gradient‐Domain Volumetric Rendering

Deep convolutional reconstruction for gradient-domain rendering

Contact Info

Product

Resources

About