Counterexamples for optimal scaling of Metropolis–Hastings chains with rough target densities

Abstract

For sufficiently smooth targets of product form it is known that the variance of a single coordinate of the proposal in RWM (random walk Metropolis) and MALA (Metropolis adjusted Langevin algorithm) should optimally scale as n −1 and as n − 1 3 with dimension n, and that the acceptance rates should be tuned to 0.234 and 0.574. We establish counterexamples to demonstrate that smoothness assumptions of the order of C 1 (R) for RWM and C 3 (R) for MALA are indeed required if these scaling rates are to hold. The counterexamples identify classes of marginal targets for which these guidelines are violated, obtained by perturbing a standard normal density (at the level of the potential for RWM and the second derivative of the potential for MALA) using roughness generated by a path of fractional Brownian motion with Hurst exponent H . For such targets there is strong evidence that RWM and MALA proposal variances should optimally be scaled as n − 1 H and as n − 1 2+H and will then obey anomalous acceptance rate guidelines. Useful heuristics resulting from this theory are discussed. The paper develops a framework capable of tackling optimal scaling results for quite general Metropolis-Hastings algorithms (possibly depending on a random environment).

Optimal design of the Barker proposal and other locally-balanced Metropolis-Hastings algorithms

“…The concept of a log-Metropolis-Hastings random variable will be crucial for our analysis of optimal scaling. We recall some key results here, for more detail see Section 3 of [21].…”

“…, and ρ n is the log-Metropolis-Hastings random variable associated with f and Q σn . The following is established in [21].…”

“…We conduct numerical experiments to verify the theory in Section 5, and provide a discussion in Section 6. Our theoretical results build on the recently introduced optimal scaling framework of [21,24]. One powerful aspect of this approach is the ability to analyze fairly generic schemes without requiring overly case-specific calculations (e.g.…”

Locally Scaled and Stochastic Volatility Metropolis– Hastings Algorithms

“…In practice, typically, the scale matrix is treated as being diagonal, with all the dimensions having the same standard deviation σ [22]. The MH algorithm can be made arbitrarily poor by making σ either very small or very large [22,23]. Roberts and Rosenthal [22] attempt to find the optimal σ based on assuming a specific form for the distribution [23], and targeting specific acceptance rates-which are related to the efficiency of the algorithm.…”

“…The MH algorithm can be made arbitrarily poor by making σ either very small or very large [22,23]. Roberts and Rosenthal [22] attempt to find the optimal σ based on assuming a specific form for the distribution [23], and targeting specific acceptance rates-which are related to the efficiency of the algorithm. Yang et al [21] extend this to more general target distributions.…”