Generalized resampled importance sampling

Abstract

As scenes become ever more complex and real-time applications embrace ray tracing, path sampling algorithms that maximize quality at low sample counts become vital. Recent resampling algorithms building on Talbot et al.'s [2005] resampled importance sampling (RIS) reuse paths spatiotemporally to render surprisingly complex light transport with a few samples per pixel. These reservoir-based spatiotemporal importance resamplers (ReSTIR) and their underlying RIS theory make various assumptions, including sample independence. But sample reuse introduces correlation , so ReSTIR-style iterative reuse loses most convergence guarantees that RIS theoretically provides. We introduce generalized resampled importance sampling (GRIS) to extend the theory, allowing RIS on correlated samples, with unknown PDFs and taken from varied domains. This solidifies the theoretical foundation, allowing us to derive variance bounds and convergence conditions in ReSTIR-based samplers. It also guides practical algorithm design and enables advanced path reuse between pixels via complex shift mappings. We show a path-traced resampler (ReSTIR PT) running interactively on complex scenes, capturing many-bounce diffuse and specular lighting while shading just one path per pixel. With our new theoretical foundation, we can also modify the algorithm to guarantee convergence for offline renderers.

Decorrelating ReSTIR Samplers via MCMC Mutations

“…All results use a GeForce RTX 3090 GPU at 1920 × 1080 resolution. Our implementation uses the settings (e.g., spatial neighborhood size and reuse radius) proposed in Bitterli et al [2020] and Lin et al [2022] for direct and indirect illumination, with the exception of 𝑀 cap = 50 in our ReSTIR PT tests.…”

“…MIS weights play an important role in ReSTIR. Beyond reducing noise in the resampling weights, they also remove bias when the supports of the source and target distributions do not match integrand 𝑓 's support (see Section 4 in Bitterli et al [2020] and Section 5 in Lin et al [2022] for further details).…”

“…This assumption may no longer hold when reusing spatially or temporally across an image (as in ReSTIR), and depends on the integral formulation used for the rendering equation. For instance, ReSTIR applied to global illumination [Ouyang et al 2021;Lin et al 2022] generates samples from PDFs with respect to solid angle. Reuse across pixels therefore requires a change of integration domain, necessitating a correction term in the resampling weights [Ouyang et al 2021, Equation 11].…”

“…Recent work by Lin et al [2022] generalizes RIS to use candidate samples 𝑦 𝑗 originating from different domains Ω 𝑗 . It achieves this via shift mapping, i.e., a bijective transformation of samples from one pixel to corresponding samples on another pixel [Lehtinen et al 2013].…”

“…Recent sampling algorithms for real-time ray tracing achieve massive speedups in scenes with complex illumination by sharing samples spatially within an image and temporally across frames [Bitterli et al 2020;Ouyang et al 2021;Lin et al 2021Lin et al , 2022. These so-called…”