Deep radiance caching: Convolutional autoencoders deeper in ray tracing

Abstract

Rendering realistic images with global illumination is a computationally demanding task and often requires dedicated hardware for feasible runtime. Recent research uses Deep Neural Networks to predict indirect lighting on image level, but such methods are commonly limited to diffuse materials and require training on each scene. We present Deep Radiance Caching (DRC), an efficient variant of Radiance Caching utilizing Convolutional Autoencoders for rendering global illumination. DRC employs a denoising neural network with Radiance Caching to support a wide range of material types, without the requirement of offline pre-computation or training for each scene. This offers high performance CPU rendering for maximum accessibility. Our method has been evaluated on interior scenes, and is able to produce high-quality images within 180 seconds on a single CPU.

Real-time neural radiance caching for path tracing

“…Neural techniques. Neural networks are capable of approximating various visual phenomena remarkably well, whether they operate in screen space [Nalbach et al 2017] or in world space, whether they are pre-trained over multiple scenes [Hermosilla et al 2019;Jiang and Kainz 2021;Kallweit et al 2017;Nalbach et al 2017] or fit to single scene [Keller and Dahm 2019;Mildenhall et al 2020;Müller et al 2020;Ren et al 2013]. The latter approaches are most closely related to ours.…”

A survey on deep learning-based Monte Carlo denoising

“…Instead of using light-field-space features for imagespace denoising, another category of research aims to directly reconstruct the denoised incident radiance field, i.e., the local light field at each pixel, for advanced goals such as unbiased path guiding [40][41][42]. We cover such works in Section 5.4.…”

“…Denoised radiance fields can also be directly integrated into pixel colors for biased rendering [42]. The method uses an autoencoder neural network to denoise low-sample radiance caches for rendering indirect illumination, and then progressively increases samples to refine the radiance caches.…”

N-SfC: Robust and Fast Shape Estimation from Caustic Images

“…With the rise of neural-network-based image processing, state-of-the-art denoising methods often integrate learned components into the processing pipeline. One can categorize these methods, based on whether they operate on the final output image [3,6,13,36] or act deeper in the path tracing process to predict global illumination effects [24] high-resolution radiance maps from low resolution samples [12]. Our method directly operates in image-space, however we note that noise statistics of photon mapping methods differs from regular unidirectional path-tracing due to the bias in the estimator [41,45].…”