This paper defines an approximation scheme for a solution of the Poisson equation of a geometrically ergodic Metropolis-Hastings chain Φ. The scheme is based on the idea of weak approximation and gives rise to a natural sequence of control variates for the ergodic average S k (F ) = (1/k) k i=1 F (Φi), where F is the force function in the Poisson equation. The main results show that the sequence of the asymptotic variances (in the CLTs for the control-variate estimators) converges to zero and give a rate of this convergence. Numerical examples in the case of a double-well potential are discussed.

For sufficiently smooth targets of product form it is known that the variance of a single coordinate of the proposal in RWM (random walk Metropolis) and MALA (Metropolis adjusted Langevin algorithm) should optimally scale as n −1 and as n − 1 3 with dimension n, and that the acceptance rates should be tuned to 0.234 and 0.574. We establish counterexamples to demonstrate that smoothness assumptions of the order of C 1 (R) for RWM and C 3 (R) for MALA are indeed required if these scaling rates are to hold. The counterexamples identify classes of marginal targets for which these guidelines are violated, obtained by perturbing a standard normal density (at the level of the potential for RWM and the second derivative of the potential for MALA) using roughness generated by a path of fractional Brownian motion with Hurst exponent H . For such targets there is strong evidence that RWM and MALA proposal variances should optimally be scaled as n − 1 H and as n − 1 2+H and will then obey anomalous acceptance rate guidelines. Useful heuristics resulting from this theory are discussed. The paper develops a framework capable of tackling optimal scaling results for quite general Metropolis-Hastings algorithms (possibly depending on a random environment).

SUMMARY We study the class of first-order locally-balanced Metropolis–Hastings algorithms introduced in Livingstone & Zanella (2022). To choose a specific algorithm within the class the user must select a balancing function g: ℝ + → ℝ + satisfying g(t) = tg(1/t), and a noise distribution for the proposal increment. Popular choices within the class are the Metropolis-adjusted Langevin algorithm and the recently introduced Barker proposal. We first establish a general limiting optimal acceptance rate of 57% and scaling of n−1/3 as the dimension n tends to infinity among all members of the class under mild smoothness assumptions on g and when the target distribution for the algorithm is of the product form. In particular we obtain an explicit expression for the asymptotic efficiency of an arbitrary algorithm in the class, as measured by expected squared jumping distance. We then consider how to optimize this expression under various constraints. We derive an optimal choice of noise distribution for the Barker proposal, optimal choice of balancing function under a Gaussian noise distribution, and optimal choice of first-order locally-balanced algorithm among the entire class, which turns out to depend on the specific target distribution. Numerical simulations confirm our theoretical findings and in particular show that a bi-modal choice of noise distribution in the Barker proposal gives rise to a practical algorithm that is consistently more efficient than the original Gaussian version.

No abstract

We study the class of first-order locally-balanced Metropolis-Hastings algorithms introduced in [9]. To choose a specific algorithm within the class the user must select a balancing function g : R → R satisfying g(t) = tg(1/t), and a noise distribution for the proposal increment. Popular choices within the class are the Metropolis-adjusted Langevin algorithm and the recently introduced Barker proposal. We first establish a universal limiting optimal acceptance rate of 57% and scaling of n −1/3 as the dimension n tends to infinity among all members of the class under mild smoothness assumptions on g and when the target distribution for the algorithm is of the product form. In particular we obtain an explicit expression for the asymptotic efficiency of an arbitrary algorithm in the class, as measured by expected squared jumping distance. We then consider how to optimise this expression under various constraints. We derive an optimal choice of noise distribution for the Barker proposal, optimal choice of balancing function under a Gaussian noise distribution, and optimal choice of first-order locally-balanced algorithm among the entire class, which turns out to depend on the specific target distribution. Numerical simulations confirm our theoretical findings and in particular show that a bi-modal choice of noise distribution in the Barker proposal gives rise to a practical algorithm that is consistently more efficient than the original Gaussian version.