Stability of adversarial Markov chains, with an application to adaptive MCMC algorithms

Abstract

We consider whether ergodic Markov chains with bounded step size remain bounded in probability when their transitions are modified by an adversary on a bounded subset. We provide counterexamples to show that the answer is no in general, and prove theorems to show that the answer is yes under various additional assumptions. We then use our results to prove convergence of various adaptive Markov chain Monte Carlo algorithms.1. Introduction. This paper considers whether bounded modifications of stable Markov chains remain stable. Specifically, we let P be a fixed time-homogeneous ergodic Markov chain kernel with bounded step size, and let {X n } be a stochastic process which follows the transition probabilities P except on a bounded subset K where an "adversary" can make arbitrary bounded jumps. Under what conditions must such a process {X n } be bounded in probability?One might think that such boundedness would follow easily, at least under mild regularity and continuity assumptions, that is, that modifying a stable continuous Markov chain inside a bounded set K couldn't possibly lead to unstable behavior out in the tails. In fact the situation is rather more subtle, as we explore herein. We will provide counterexamples to show that boundedness may fail even for well-behaved continuous chains. We will then show that under various additional conditions, including bounds on transition probabilities and/or small set assumptions and/or geometric ergodicity, such boundedness does hold.

Adaptive Incremental Mixture Markov Chain Monte Carlo

“…This result is a consequence of Theorem 2 of Roberts and Rosenthal (2007) combined with the recent developments in Craiu et al (2015) and Rosenthal and Yang (2017). The proof is given in Appendix B.…”

Adaptive Component-Wise Multiple-Try Metropolis Sampling

“…We proceed similarly to the proof of Proposition 23 of Craiu et al (2015). By our assumption (A1), the process {X n } is bounded in probability, in fact X n ≤ L for all n.…”

“…The Containment condition of Roberts and Rosenthal (2007) (see also Craiu et al (2015); Rosenthal and Yang (2016) states that the process's convergence times are bounded in probability, i.e. that {M (X n , Γ n )} ∞ n=1 is bounded in probability, where M (x, γ) := inf{n ≥ 1 : P n γ (x, ·) − π(·) ≤ } for all > 0, and P n γ is a fixed n-step proposal kernel.…”

A framework for adaptive MCMC targeting multimodal distributions

“…condition e) below). Adapting on a compact set has been theoretically investigated in [16] and used in certain adaptive Gibbs sampler contexts in [13]. We shall use Lemma 3.5 as the main tool for establishing ergodic theorems for JAMS.…”

“…Sensor network localisation. Starting points for the BFGS procedures were sampled uniformly on [0, 1] 16 . The number of function and gradient evaluations for these runs varied between 175 and 876, with an average of 400.…”