Real-Time Denoising of Volumetric Path Tracing for Direct Volume Rendering

Abstract

Direct Volume Rendering (DVR) using Volumetric Path Tracing (VPT) is a scientific visualization technique that simulates light transport with objects' matter using physically-based lighting models. Monte Carlo (MC) path tracing is often used with surface models, yet its application for volumetric models is difficult due to the complexity of integrating MC light-paths in volumetric media with none or smooth material boundaries. Moreover, auxiliary geometry-buffers (G-buffers) produced for volumes are typically very noisy, failing to guide image denoisers relying on that information to preserve image details. This makes existing real-time denoisers, which take noise-free G-buffers as their input, less effective when denoising VPT images. We propose the necessary modifications to an image-based denoiser previously used when rendering surface models, and demonstrate effective denoising of VPT images. In particular, our denoising exploits temporal coherence between frames, without relying on noise-free G-buffers, which has been a common assumption of existing denoisers for surface-models. Our technique preserves high-frequency details through a weighted recursive least squares that handles heterogeneous noise for volumetric models. We show for various real data sets that our method improves the visual fidelity and temporal stability of VPT during classic DVR operations such as camera movements, modifications of the light sources, and editions to the volume transfer function.

State‐of‐the‐art in Large‐Scale Volume Visualization Beyond Structured Data

“…Representations optimized for volume rendering are sometimes also used for arbitrary sampling, e.g., to compute flow visualizations [ZSM * 22], which involves more divergent memory access patterns. Finally, we observe a recent trend that path tracing is adopted by the sci‐vis community [MSG * 22,ZWS * 22b], which is most likely driven by the recent availability of high‐quality denoising techniques [IGMM22] and requires sample placement at arbitrary positions. Taking individual samples becomes even more costly with this approach, so efficient volume traversal data structures become even more critical.…”

“…In recent years, we have observed a gradual shift towards volumetric path tracing, also in the scientific visualization community [KPB12, WJA * 17, MHK * 19, HMES20, IGMM22, MSG * 22, ZWB * 22, XTC * ss, ZWS * 22b]. Instead of stepwise sampling the domain [ t min , t max ] using ray marching, volumetric path tracers compute free‐flight distances inside the domain.…”

Volumetric Multi‐View Rendering

“…Such combination could further reduce the sample inhomogeneity in multi‐view images. Similarly, image‐based denoising techniques have been proposed to further reduce noise in the context of spatio‐temporal sequences [VRM * 18, ZRJ * 15] for surfaces and volumes [IGMM21]. While they often require exporting numerous additional feature layers per image, our technique could feed these denoisers with cleaner inputs for improved results.…”

Combined volume and surface rendering with global illumination caching

“…Even with the rapid development of GPU technology, realtime rendering of complex volumetric data with an unbiased physically based approach is not possible without additional steps such as denoising presented by Iglesias-Guietan et al [17], super-resolution presented by Weiss et al [51] or global illumination caching, such as radiance caching by Jarosz et al [19], irradiance caching by Ribardieere et al [40] and Khlebnikov et al [24], and transmittance caching by Weber et al [50]. In our approach, we implemented irradiance caching for computing a global illumination volume, allowing us to use faster ray casting techniques to render the data while also displaying the isosurfaces.…”