Offline Deep Importance Sampling for Monte Carlo Path Tracing

Abstract

Although modern path tracers are successfully being applied to many rendering applications, there is considerable interest to push them towards ever‐decreasing sampling rates. As the sampling rate is substantially reduced, however, even Monte Carlo (MC) denoisers–which have been very successful at removing large amounts of noise–typically do not produce acceptable final results. As an orthogonal approach to this, we believe that good importance sampling of paths is critical for producing better‐converged, path‐traced images at low sample counts that can then, for example, be more effectively denoised. However, most recent importance‐sampling techniques for guiding path tracing (an area known as “path guiding”) involve expensive online (per‐scene) training and offer benefits only at high sample counts. In this paper, we propose an offline, scene‐independent deep‐learning approach that can importance sample first‐bounce light paths for general scenes without the need of the costly online training, and can start guiding path sampling with as little as 1 sample per pixel. Instead of learning to “overfit” to the sampling distribution of a specific scene like most previous work, our data‐driven approach is trained a priori on a set of training scenes on how to use a local neighborhood of samples with additional feature information to reconstruct the full incident radiance at a point in the scene, which enables first‐bounce importance sampling for new test scenes. Our solution is easy to integrate into existing rendering pipelines without the need for retraining, as we demonstrate by incorporating it into both the Blender/Cycles and Mitsuba path tracers. Finally, we show how our offline, deep importance sampler (ODIS) increases convergence at low sample counts and improves the results of an off‐the‐shelf denoiser relative to other state‐of‐the‐art sampling techniques.

A survey on deep learning-based Monte Carlo denoising

“…Instead of using light-field-space features for imagespace denoising, another category of research aims to directly reconstruct the denoised incident radiance field, i.e., the local light field at each pixel, for advanced goals such as unbiased path guiding [40][41][42]. We cover such works in Section 5.4.…”

“…Bako et al [40] noted that even modern deep learning-based MC denoisers do not produce acceptable final results for high-quality rendering, and turned to the recent path guiding techniques that aim to predict the incident radiance field at each pixel, which enables use of a guided probability distribution function (PDF) for first-bounce importance sampling. While existing path guiding approaches involve expensive online learning and offer benefits only at high sample counts, the authors proposed an offline, scene-independent deep learning-based approach that can importance-sample first-bounce light paths for general scenes.…”

Hierarchical neural reconstruction for path guiding using hybrid path and photon samples

“…However, the online learning process is relatively slow, which results in noisy sampling maps for a long time, restricting the guiding efficiency. Bako et al [2019] leverages offline deep learning, but it can only guide the first bounce, which naturally cannot outperform traditional online methods for such a scene with strong global illumination. Ruppert et al [2020] introduces parallax compensation and uses mixture models (VMMs) to represent sampling distributions.…”

“…Previous methods [Müller et al 2017;Rath et al 2020;Ruppert et al 2020] often require a slow online learning process to obtain accurate sampling distributions for path guiding. While some recent works [Bako et al 2019;Zhu et al 2020b] use offline-trained neural networks, their methods require large system memory and can only reconstruct sampling distributions at a low resolution, restricting the accuracy and efficiency of path guiding.…”

Two‐stage Resampling for Bidirectional Path Tracing with Multiple Light Sub‐paths

“…Path guiding : Path guiding is an adaptive importance sampling method that learns the distributions of the incoming radiance or the importance prior to sampling. The precomputed distributions are represented by a Gaussian mixture model (GMM) [VKv∗14, HEV∗16], an adaptive tree structure [MGN17], or deep neural networks [MMR∗19, BMDS19]. Since these methods are orthogonal to our work, path guiding can be combined with our method for efficient sampling of sub‐paths.…”