Randomized Quasi-Monte Carlo Simulation of Markov Chains with an Ordered State Space

Abstract

Summary. We study a randomized quasi-Monte Carlo method for estimating the state distribution at each step of a Markov chain with totally ordered (discrete or continuous) state space. The number of steps in the chain can be random and unbounded. The method can be used in particular to get a low-variance unbiased estimator of the expected total cost up to some random stopping time, when statedependent costs are paid at each step. We provide numerical illustrations where the variance reduction with respect to standard Monte Carlo is substantial.

Efficient computation of transient solutions of the chemical master equation based on uniformization and quasi-Monte Carlo

“…It is shown that uniformization, combined with a QMC method, 19 can be efficiently applied to the simulation of biochemical reaction networks when the aim is to compute transient solutions of the PDF. Two numerical examples have been considered, and the QMC method yields superior results.…”

“…19 that the method yields unbiased estimators of averages and empirical variances. 19 that the method yields unbiased estimators of averages and empirical variances.…”

“…16,17,19 For a review see Ref. 19 uses a single randomized quasirandom sequence and can easily be applied to simulation with a large number of steps. Most of the methods use a sequence of dimensionality dN T to simulate the discrete time chain N T steps if the transitions require d uniform random numbers.…”

“…Most of the methods use a sequence of dimensionality dN T to simulate the discrete time chain N T steps if the transitions require d uniform random numbers. 19 to SSA will be as follows. In addition, the generation of high dimensional sequences is expensive.…”

What Monte Carlo models can do and cannot do efficiently?

“…and adding it to all the points of the sequence (modulo 1) [42] (see, also [43,44]). The integral is thus approximated via…”