Exact estimation for Markov chain equilibrium expectations

Abstract

We introduce a new class of Monte Carlo methods, which we call exact estimation algorithms.Such algorithms provide unbiased estimators for equilibrium expectations associated with realvalued functionals defined on a Markov chain. We provide easily implemented algorithms for the class of positive Harris recurrent Markov chains, and for chains that are contracting on average. We further argue that exact estimation in the Markov chain setting provides a significant theoretical relaxation relative to exact simulation methods.

Unbiased Markov Chain Monte Carlo Methods with Couplings

“…We congratulate Jacob, O’Leary and Atchadé for their inspiring work, which elaborates on the general Markov chain debiasing method of Glynn and Rhee () and demonstrates how it can be applied in practice in various settings. We would like to highlight the importance of understanding the scalability properties of the related couplings, in particular with respect to increasing model dimension, to assess the applicability of the method for challenging problems for which inference is difficult with existing techniques.…”

“…From a practical point of view, the coupling approach of Jacob and his colleagues is easier to put into practice than the random‐truncation technique of Glynn and Rhee () in this context. Indeed, as long as the likelihood of the observations given the latent states is bounded for state space models or the likelihood is bounded for general posterior targets, then the PIMH algorithm is uniformly ergodic for any N and, under very weak additional assumptions, the coupling procedure returns finite variance estimates in finite expected time.…”

“…Indeed, as long as the likelihood of the observations given the latent states is bounded for state space models or the likelihood is bounded for general posterior targets, then the PIMH algorithm is uniformly ergodic for any N and, under very weak additional assumptions, the coupling procedure returns finite variance estimates in finite expected time. To obtain similar guarantees for the methodology of Glynn and Rhee (), quantitative convergence results for the PIMH algorithm would need to be obtained to select an appropriate distribution for the truncation time.…”

“…Middleton, Deligiannidis and Doucet emphasize that the approach proposed does not involve random truncation variables, in contrast with for example Glynn and Rhee () and Agapiou et al . ().…”

“…() for an example of estimation of such a contraction rate, in the case of a coupled Metropolis–Hastings algorithm on a logistic regression with three covariates. Removing the need for truncation variables by considering chains that meet exactly appears both as a straightforward modification of Glynn and Rhee () and as a step forward in terms of applicability.…”