Variance reduction for Metropolis–Hastings samplers

Abstract

We introduce a general framework that constructs estimators with reduced variance for random walk Metropolis and Metropolis-adjusted Langevin algorithms. The resulting estimators require negligible computational cost and are derived in a post-process manner utilising all proposal values of the Metropolis algorithms. Variance reduction is achieved by producing control variates through the approximate solution of the Poisson equation associated with the target density of the Markov chain. The proposed method is based on approximating the target density with a Gaussian and then utilising accurate solutions of the Poisson equation for the Gaussian case. This leads to an estimator that uses two key elements: (1) a control variate from the Poisson equation that contains an intractable expectation under the proposal distribution, (2) a second control variate to reduce the variance of a Monte Carlo estimate of this latter intractable expectation. Simulated data examples are used to illustrate the impressive variance reduction achieved in the Gaussian target case and the corresponding effect when target Gaussianity assumption is violated. Real data examples on Bayesian logistic regression and stochastic volatility models verify that considerable variance reduction is achieved with negligible extra computational cost.

Regularized Zero-Variance Control Variates

“…where α(x, y) = min(1, R(x, y)) is the acceptance probability, R(x, y) = π(y)q(x|y) π(x)q(y|x) is the MH-ratio and q(y|x) is the density of the proposal distribution evaluated at y given current value x. This is not analytically tractable but it is feasible to get an unbiased estimate by storing and using the proposals and acceptance probabilities from each step in the chain (Alexopoulos et al, 2023). If X (i+1) * ∼ q(•|X (i) ) denotes the proposed value moving from X (i) to X (i+1) , then the estimator could be written as…”

“…High variance in unbiased estimates for u(x) could negatively affect the performance of the control variates and unfortunately advice in this direction is limited in the literature. To keep the variance of the aforementioned control variate down, Alexopoulos et al (2023) introduce an additional control variate and Tsourti (2012) use multiple importance sampling. In the absence of low-variance unbiased estimators for h − P h, it may be better to stick with control variates based on analytically tractable P .…”

“…A challenge with this approach is that approximation of P for the partitioned space requires Monte Carlo methods with further evaluations of π (up to a normalising constant). Alexopoulos et al (2023) developed an approach for estimating marginal means from certain MH samplers using numerical integration to approximately solve Poisson's equation for a simpler problem based on a Gaussian approximation to π.…”