Optimal multiple importance sampling

Kondapaneni, Ivo; Vévoda, Petr; Grittmann, Pascal; Skřivan, Tomáš; Slusallek, Philipp; Křivánek, Jaroslav

doi:10.1145/3306346.3323009

Cited by 38 publications

(25 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ideally, we hope these weighting functions can minimize the sum of variance at each position. Formally, we have Optimizing weighting functions to minimize the variance of ordinary MIS estimator has been studied by a recent work (Kondapaneni et al, 2019). Our setting is different from it in that (1) we focus on self-normalized MIS and that (2) the function f (•) is vector-valued instead of scalar-valued.…”

Section: Analysis On the Optimal Weighting Function In Multiple Impor...mentioning

confidence: 99%

Linear Complexity Randomized Self-attention Mechanism

Zheng¹,

Wang²,

Kong³

2022

Preprint

View full text Add to dashboard Cite

Recently, random feature attentions (RFAs) are proposed to approximate the softmax attention in linear time and space complexity by linearizing the exponential kernel. In this paper, we first propose a novel perspective to understand the bias in such approximation by recasting RFAs as self-normalized importance samplers. This perspective further sheds light on an unbiased estimator for the whole softmax attention, called randomized attention (RA). RA constructs positive random features via query-specific distributions and enjoys greatly improved approximation fidelity, albeit exhibiting quadratic complexity. By combining the expressiveness in RA and the efficiency in RFA, we develop a novel linear complexity self-attention mechanism called linear randomized attention (LARA). Extensive experiments across various domains demonstrate that RA and LARA significantly improve the performance of RFAs by a substantial margin.

show abstract

Section: Analysis On the Optimal Weighting Function In Multiple Impor...mentioning

confidence: 99%

Linear Complexity Randomized Self-attention Mechanism

Zheng¹,

Wang²,

Kong³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The balance heuristic is also present in most of successful adaptive IS (AIS) methods, see [ 11 , 12 , 13 , 14 , 15 ], in particular in the case where all techniques are used to simulate the same number of samples. Recently, MIS has returned to the main focus in computer graphics in [ 16 ], where it is shown that allowing weights to be negative, the reduction over balance heuristics of the resulting MIS estimator can be higher than the one predicted by Veach bounds [ 17 ]; in [ 18 ], where one of the sampling techniques is optimized to decrease the overall variance of the resulting MIS estimator; in [ 19 ], where the weights are made proportional to the quotients of second moments divided by the variances of the independent techniques; in [ 20 ], where MIS is generalized to uncountably infinite sets of techniques.…”

Section: Introductionmentioning

confidence: 99%

Generalizing the Balance Heuristic Estimator in Multiple Importance Sampling

Sbert

Elvira

2022

Entropy

View full text Add to dashboard Cite

In this paper, we propose a novel and generic family of multiple importance sampling estimators. We first revisit the celebrated balance heuristic estimator, a widely used Monte Carlo technique for the approximation of intractable integrals. Then, we establish a generalized framework for the combination of samples simulated from multiple proposals. Our approach is based on considering as free parameters both the sampling rates and the combination coefficients, which are the same in the balance heuristics estimator. Thus our novel framework contains the balance heuristic as a particular case. We study the optimal choice of the free parameters in such a way that the variance of the resulting estimator is minimized. A theoretical variance study shows the optimal solution is always better than the balance heuristic estimator (except in degenerate cases where both are the same). We also give sufficient conditions on the parameter values for the new generalized estimator to be better than the balance heuristic estimator, and one necessary and sufficient condition related to χ2 divergence. Using five numerical examples, we first show the gap in the efficiency of both new and classical balance heuristic estimators, for equal sampling and for several state of the art sampling rates. Then, for these five examples, we find the variances for some notable selection of parameters showing that, for the important case of equal count of samples, our new estimator with an optimal selection of parameters outperforms the classical balance heuristic. Finally, new heuristics are introduced that exploit the theoretical findings.

show abstract

“…Another venue would be to couple it with progressive photon mapping [8,9] or vertex merging techniques [10]. Yet another potential area is coupling it with the latest results on improved multiple importance sampling [15,19,29]. Finally, it would be interesting to automate the choice of the constant in our biased estimator to minimize total error, seen as the sum of bias and variance.…”

Section: Future Workmentioning

confidence: 99%

Online Path Sampling Control with Progressive Spatio-temporal Filtering

Pantaleoni¹

2020

SN COMPUT. SCI.

View full text Add to dashboard Cite

This work introduces a novel technique called progressive spatio-temporal filtering, an efficient method to build all-frequency approximations to the light transport distribution into a scene by filtering individual samples produced by an underlying path sampler, using online, iterative algorithms and data-structures that exploit both the spatial and temporal coherence of the approximated light field. Unlike previous approaches, the proposed method is both more efficient, due to its use of an iterative temporal feedback loop that massively improves convergence to a noise-free approximant, and more flexible, due to its introduction of a spatio-directional hashing representation that allows to encode directional variations like those due to glossy reflections. We then introduce four different methods to employ the resulting approximations to control the underlying path sampler and/or modify its associated estimator, greatly reducing its variance and enhancing its robustness to complex lighting scenarios. The core algorithms are highly scalable and low-overhead, requiring only minor modifications to an existing path tracer. Combined with the most recent advances in ray-tracing hardware and denoising algorithms, our work enables performing high-quality light transport simulations including complex visibility and transport phenomena in real time.

show abstract

Optimal multiple importance sampling

Cited by 38 publications

References 27 publications

Linear Complexity Randomized Self-attention Mechanism

Linear Complexity Randomized Self-attention Mechanism

Generalizing the Balance Heuristic Estimator in Multiple Importance Sampling

Online Path Sampling Control with Progressive Spatio-temporal Filtering

Contact Info

Product

Resources

About