Sample-based Monte Carlo denoising using a kernel-splatting network

Gharbi, Michaël; Li, Tzu‐Mao; Aittala, Miika; Lehtinen, Jaakko; Durand, Frédo

doi:10.1145/3306346.3322954

Cited by 81 publications

(82 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Vogels et al [2018] extended the kernel prediction strategy in the work [Bako et al 2017] by also considering temporal coherence at multiple scales. In concurrent works, Kettunen et al [2019] attempted reconstruction for gradient-domain rendering, whilst Gharbi et al [2019] utilized raw Monte Carlo samples as high-order statistics and a novel splatting approach to achieve better results with larger computational cost and storage space though.…”

Section: Related Workmentioning

confidence: 99%

Adversarial Monte Carlo denoising with conditioned auxiliary feature modulation

Xu¹,

Zhang²,

Wang

et al. 2019

ACM Trans. Graph.

View full text Add to dashboard Cite

including regression-based and learning-based methods, have been explored to achieve better rendering quality with less computational cost. However, most of these methods rely on handcrafted optimization objectives, which lead to artifacts such as blurs and unfaithful details. In this paper, we present an adversarial approach for denoising Monte Carlo rendering. Our key insight is that generative adversarial networks can help denoiser networks to produce more realistic high-frequency details and global illumination by learning the distribution from a set of high-quality Monte Carlo path tracing images. We also adapt a novel feature modulation method to utilize auxiliary features better, including normal, albedo and depth. Compared to previous state-of-the-art methods, our approach produces a better reconstruction of the Monte Carlo integral from a few samples, performs more robustly at different sample rates, and takes only a second for megapixel images.

show abstract

Section: Related Workmentioning

confidence: 99%

Adversarial Monte Carlo denoising with conditioned auxiliary feature modulation

Xu¹,

Zhang²,

Wang

et al. 2019

ACM Trans. Graph.

View full text Add to dashboard Cite

show abstract

“…A generative adversarial network (GAN) MC denoising method was proposed by Xu et al [XZW19] to achieve higher perceptual quality. Gharbi et al [GLA∗19] proposed a sample‐based denoising method (SBMC), by splatting each sample onto nearby pixels to produce denoised results which allows very low sampling rate.…”

Section: Related Workmentioning

confidence: 99%

Unsupervised Image Reconstruction for Gradient‐Domain Volumetric Rendering

Xu¹,

Sun

Wang³

et al. 2020

Computer Graphics Forum

View full text Add to dashboard Cite

Gradient-domain rendering can highly improve the convergence of light transport simulation using the smoothness in image space. These methods generate image gradients and solve an image reconstruction problem with rendered image and the gradient images. Recently, a previous work proposed a gradient-domain volumetric photon density estimation for homogeneous participating media. However, the image reconstruction relies on traditional L1 reconstruction, which leads to obvious artifacts when only a few rendering passes are performed. Deep learning based reconstruction methods have been exploited for surface rendering, but they are not suitable for volume density estimation. In this paper, we propose an unsupervised neural network for image reconstruction of gradient-domain volumetric photon density estimation, more specifically for volumetric photon mapping, using a variant of GradNet with an encoded shift connection and a separated auxiliary feature branch, which includes volume based auxiliary features such as transmittance and photon density. Our network smooths the images on global scale and preserves the high frequency details on a small scale. We demonstrate that our network produces a higher quality result, compared to previous work. Although we only considered volumetric photon mapping, it's straightforward to extend our method for other forms, like beam radiance estimation. CCS Concepts • Computing methodologies → Neural network; Ray tracing; † Joint first author.

show abstract

“…Gharbi et al [7] applied learning directly between samples and kernel parameters, instead of starting with noisy images. Since samples include more information, it produces higher quality even with only a few samples.…”

Section: Machine Learning Based Monte Carlo Denoisingmentioning

confidence: 99%

A detail preserving neural network model for Monte Carlo denoising

Lin

Wang

et al. 2020

Comp. Visual Media

View full text Add to dashboard Cite

Monte Carlo based methods such as path tracing are widely used in movie production. To achieve low noise, they require many samples per pixel, resulting in long rendering time. To reduce the cost, one solution is Monte Carlo denoising, which renders the image with fewer samples per pixel (as little as 128) and then denoises the resulting image. Many Monte Carlo denoising methods rely on deep learning: they use convolutional neural networks to learn the relationship between noisy images and reference images, using auxiliary features such as position and normal together with image color as inputs. The network predicts kernels which are then applied to the noisy input. These methods show powerful denoising ability, but tend to lose geometric or lighting details and to blur sharp features during denoising.In this paper, we solve this issue by proposing a novel network structure, a new input feature-light transport covariance from path space-and an improved loss function. Our network separates feature buffers from the color buffer to enhance detail effects. The features are extracted separately and then integrated into a shallow kernel predictor. Our loss function considers perceptual loss, which also improves detail preservation. In addition, we use a light transport covariance feature in path space as one of the features, which helps to preserve illumination details. Our method denoises Monte Carlo path traced images while preserving details much better than previous methods.

show abstract

Sample-based Monte Carlo denoising using a kernel-splatting network

Cited by 81 publications

References 47 publications

Adversarial Monte Carlo denoising with conditioned auxiliary feature modulation

Adversarial Monte Carlo denoising with conditioned auxiliary feature modulation

Unsupervised Image Reconstruction for Gradient‐Domain Volumetric Rendering

A detail preserving neural network model for Monte Carlo denoising

Contact Info

Product

Resources

About