Robust Image Denoising Using a Virtual Flash Image for Monte Carlo Ray Tracing

Samples with high contribution but low probability density, often called fireflies, occur in all practical Monte Carlo estimators and are part of computing unbiased estimates. For finite‐sample estimates, however, they can lead to excessive variance. Rejecting all samples classified as outliers, as suggested in previous work, leads to estimates that are too low and can cause undesirable artefacts. In this paper, we show how samples can be re‐weighted depending on their contribution and sampling frequency such that the finite‐sample estimate gets closer to the correct expected value and the variance can be controlled. For this, we first derive a theory for how samples should ideally be re‐weighted and that this would require the probability density function of the optimal sampling strategy. As this probability density function is generally unknown, we show how the discrepancy between the optimal and the actual sampling strategy can be estimated and used for re‐weighting in practice. We describe an efficient algorithm that allows for the necessary analysis of per‐pixel sample distributions in the context of Monte Carlo rendering without storing any individual samples, with only minimal changes to the rendering algorithm. It causes negligible runtime overhead, works in constant memory and is well suited for parallel and progressive rendering. The re‐weighting runs as a fast post‐process, can be controlled interactively and our approach is non‐destructive in that the unbiased result can be reconstructed at any time.

Section: The Cascaded Sample Count Framebuffermentioning

confidence: 99%

Re‐Weighting Firefly Samples for Improved Finite‐Sample Monte Carlo Estimates

Zirr

Hanika

Dachsbacher

2018

“…Van De Ville and Kocher [VDVK09] propose a SURE‐based error estimate for NL‐means denoising, although they used it for parameter selection on a per‐image rather than per‐pixel basis. Moon et al [MJL*13] apply an NL‐means filter guided by a virtual flash image. Our approach is most related to the work by Li et al, with some significant differences.…”

Section: Previous Workmentioning

confidence: 99%

Robust Denoising using Feature and Color Information

Rousselle

Manzi

Zwicker

2013

Figure 1: We propose a method to denoise Monte Carlo renderings using noisy color (left) and feature buffers (middle: texture, caustics, visibility, normals) as an input. We construct a denoising filter by combining color and feature information using a SURE error estimate. Our results (right) improve visually and quantitatively over the previous state-of-the-art. AbstractWe propose a method that robustly combines color and feature buffers to denoise Monte Carlo renderings. On one hand, feature buffers, such as per pixel normals, textures, or depth, are effective in determining denoising filters because features are highly correlated with rendered images. Filters based solely on features, however, are prone to blurring image details that are not well represented by the features. On the other hand, color buffers represent all details, but they may be less effective to determine filters because they are contaminated by the noise that is supposed to be removed. We propose to obtain filters using a combination of color and feature buffers in an NL-means and cross-bilateral filtering framework. We determine a robust weighting of colors and features using a SURE-based error estimate. We show significant improvements in subjective and quantitative errors compared to the previous state-of-the-art. We also demonstrate adaptive sampling and space-time filtering for animations.

“…State‐of‐the‐art denoising algorithms all rely on a combination of noisy colour and feature information to guide the denoising process. Our work builds on the joint NL‐Means filter used in previous works [RMZ13, MJL*13, KBS15, ZRJ*15], which we describe in this section. Our generalization to deep data follows in Section 4.…”

Section: Denoising Flat Images With Nl‐meansmentioning

confidence: 99%

Denoising Deep Monte Carlo Renderings

Vicini

Adler

Novák

et al. 2018

We present a novel algorithm to denoise deep Monte Carlo renderings, in which pixels contain multiple colour values, each for a different range of depths. Deep images are a more expressive representation of the scene than conventional flat images. However, since each depth bin receives only a fraction of the flat pixel's samples, denoising the bins is harder due to the less accurate mean and variance estimates. Furthermore, deep images lack a regular structure in depth—the number of depth bins and their depth ranges vary across pixels. This prevents a straightforward application of patch‐based distance metrics frequently used to improve the robustness of existing denoising filters. We address these constraints by combining a flat image‐space non‐local means filter operating on pixel colours with a deep cross‐bilateral filter operating on auxiliary features (albedo, normal, etc.). Our approach significantly reduces noise in deep images while preserving their structure. To our best knowledge, our algorithm is the first to enable efficient deep‐compositing workflows with denoised Monte Carlo renderings. We demonstrate the performance of our filter on a range of scenes highlighting the challenges and advantages of denoising deep images.