Stochastic approximation of quasi-stationary distributions on compact spaces and applications

Abstract

As a continuation of a recent paper, dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact metric space killed in finite time. The idea is to run the process until extinction and then to bring it back to life at a position randomly chosen according to the (possibly weighted) empirical occupation measure of its past positions. General conditions are given ensuring the convergence of this measure to the quasi-stationary distribution of the chain. We then apply this method to the numerical approximation of the quasi-stationary distribution of a diffusion process killed on the boundary of a compact set. Finally, the sharpness of the assumptions is illustrated through the study of the algorithm in a non-irreducible setting.

An approximation scheme for quasi-stationary distributions of killed diffusions

“…This is not the same as the present work; we are assuming a boundaryless manifold, and instead of hard killing at a boundary, killing occurs at a smooth state-dependent rate κ as in (3). The key difference in the proposed setting of [4,Section 8.3] is that in their case, the state space is no longer compact, and hence additional arguments ensuring almost sure tightness of the empirical occupation measures are needed.…”

“…We follow generally the path mapped out by [5,4,20], often referred to as the "ODE method", cf. [2,21].…”

“…Let P(M ) denote the space of Borel probability measures on M , equipped with the topology of weak-* convergence, namely convergence along all bounded continuous test functions. (This is conventionally called simply weak convergence in probability theory, but we follow the terminology of [4] to avoid confusion when working with the Banach space C(M ) and its dual.) Thus P(M ) is a compact metrisable space, by the Prokhorov theorem, cf.…”

“…4.3.4 (5) t (s)f The final error term is a jump martingale term. We use the same approach as above for (4) t (s)f . For this jump martingale the quadratic variation is…”

Non-Markovian Monte Carlo on Directed Graphs

“…This implies that the historical empirical distributionμ t can be computed recursively based on (21). Although the convergence ofμ t to the QSD ν was shown for any arbitrary initial state Z 0 withμ 0 =δ Z 0 in [10,11], its proof based on the stochastic approximation theory and the induced differential equations in fact does not require the initial distributionμ 0 in (21) to be in the form of δ Z 0 , whereμ 0 has a value of 1 at the element Z 0 and zero at all other elements, but can be made arbitrary as long asμ 0 is a probability distribution (vector) on N. This freedom on the choice ofμ 0 gives us a great degree of flexibility. Clearly, the choice of µ 0 affects the whole evolution of the process {Z t } for all time t via redistribution mechanism, Algorithm 2 Dynamic Non-Markovian Monte Carlo 1: /* This pseudocode only replaces lines 3-10 of Algorithm 1 */ 2: Generate u 1 ,…”