Importance sampling the union of rare events with an application to power systems analysis

Abstract

We consider importance sampling to estimate the probability µ of a union of J rare events Hj defined by a random variable x. The sampler we study has been used in spatial statistics, genomics and combinatorics going back at least to Karp and Luby (1983). It works by sampling one event at random, then sampling x conditionally on that event happening and it constructs an unbiased estimate of µ by multiplying an inverse moment of the number of occuring events by the union bound. We prove some variance bounds for this sampler. For a sample size of n, it has a variance no larger than µ(μ − µ)/n whereμ is the union bound. It also has a coefficient of variation no larger than (J + J −1 − 2)/(4n) regardless of the overlap pattern among the J events. Our motivating problem comes from power system reliability, where the phase differences between connected nodes have a joint Gaussian distribution and the J rare events arise from unacceptably large phase differences. In the grid reliability problems even some events defined by 5772 constraints in 326 dimensions, with probability below 10 −22 , are estimated with a coefficient of variation of about 0.0024 with only n = 10,000 sample values.

Frequency violations from random disturbances: an MCMC approach

“…Estimating the probability of rare events in power systems is computationally challenging. Recent work in this area includes [OM17], [NZZ17a], [NZZ17b] and the present paper is complementary to such studies. Instead we aim to generate a representative sample of disturbances, conditional on a RoCoF violation occurring.…”

Multiple Importance Sampling for Efficient Symbol Error Rate Estimation

“…We follow the choice of proposals in [19]. First, the number of proposals, K, is equal to the number of hyperplanes, being each proposal a truncated version of the target distribution (a Gaussian centered at the received symbol) beyond each hyperplane, i.e., q k (x) = I S k (x)π (x) P k , where P k = ∫ I S k (x)π (x)dx is the integral of the target distribution beyond the hyperplane (the procedure for the efficient simulation from a generic truncated Gaussian distribution is described in Appendices A and B).…”

“…Let us first describe the simulation of a truncated Gaussian N (0, I) in the half-space described by x T ω ≥ τ , which first proceeds by simulating the sample x from the complementary half-space (i.e., x T ω < τ ), and then delivering the −x for numerical stability. The algorithm described in [19] proceeds as follows: 1) Simulate z ∼ N (0, I) 2) Simulate u ∼ U(0, 1) 3) Let y = Φ −1 (uΦ(−τ )), where Φ(τ ) denotes the cumulative distribution function for the standard Gaussian 4) Let x = ωy + I − ωω T z 5) Output x = −x B. Extension to a generic truncated Gaussian N (µ, Σ)…”

“…In this letter, we apply a recently proposed multiple importance sampling technique called ALOE ("At Least One rare Event") [19] to estimate the SER in additive white Gaussian noise (AWGN) channels.…”

The Square Root Rule for Adaptive Importance Sampling

“…Importance sampling is a fundamental Monte Carlo method used in finance (Glasserman, 2004), reliability (Au and Beck, 1999;Owen et al, 2019), coding theory (Elvira and Santamaria, 2019), particle transport (Kong and Spanier, 2011;Kollman et al, 1999), computer graphics (Veach and Guibas, 1995;Pharr et al, 2016), queuing (Blanchet et al, 2007), and sequential analysis (Lai and Siegmund, 1977), among other areas.…”