The Square Root Rule for Adaptive Importance Sampling

Abstract

In adaptive importance sampling, and other contexts, we have K > 1 unbiased and uncorrelated estimatesμ k of a common quantity µ. The optimal unbiased linear combination weights them inversely to their variances but those weights are unknown and hard to estimate. A simple deterministic square root rule based on a working model that Var(μ k ) ∝ k −1/2 gives an unbisaed estimate of µ that is nearly optimal under a wide range of alternative variance patterns. We show that if Var(μ k ) ∝ k −y for an unknown rate parameter y ∈ [0, 1] then the square root rule yields the optimal variance rate with a constant that is too large by at most 9/8 for any 0 y 1 and any number K of estimates. Numerical work shows that rule is similarly robust to some other patterns with mildly decreasing variance as k increases.