Role of current fluctuations in nonreversible samplers

Abstract

It is known that the distribution of nonreversible Markov processes breaking the detailed balance condition converges faster to the stationary distribution compared to reversible processes having the same stationary distribution. This is used in practice to accelerate Markov chain Monte Carlo algorithms that sample the Gibbs distribution by adding nonreversible transitions or non-gradient drift terms. The breaking of detailed balance also accelerates the convergence of empirical estimators to their ergodic expectation in the long-time limit. Here, we give a physical interpretation of this second form of acceleration in terms of currents associated with the fluctuations of empirical estimators, using the level 2.5 of large deviations, which characterises the likelihood of density and current fluctuations in Markov processes. Focusing on diffusion processes, we show that there is accelerated convergence because estimator fluctuations arise in general with current fluctuations, leading to an added large deviation "cost" compared to the reversible case, which shows no current. We study the current fluctuation most likely to arise in conjunction with a given estimator fluctuation, and provide bounds on the acceleration, based on approximations of this current. We also obtain a new orthogonality condition for fluctuating currents, which can be used to determine the accuracy of the bounds obtained. We illustrate these results for the simple diffusion on the circle and the Ornstein-Uhlenbeck process in two dimensions.

Stochastic dynamics without detailed balance condition connecting simple gradient method and Hamiltonian Monte Carlo

“…The probability current realizes a biased sampling, resulting in accelerated convergence to the target distribution [32]. It is known in such a system that the convergence of the long-time average of physical quantities, namely, the empirical average, to the ensemble average is accelerated [33].…”

Markov trajectories: Microcanonical Ensembles based on empirical observables as compared to Canonical Ensembles based on Markov generators

“…The issue (i) is now solved because the large deviations at Level 2.5 are closed and explicit for general Markov processes, including discrete-time Markov chains [3][4][5][6][7][8], continuous-time Markov jump processes [4, and Diffusion processes [7,8,12,13,17,27,28,[31][32][33][34]. The Level 2 concerning the empirical density ρ T (C) alone should be now obtained via the optimization of the Level 2.5 over the empirical flows, but this contraction is not always explicit.…”

“…of the trajectory x(0 ≤ t ≤ T ) can be rewritten as a linear combination of the empirical observables E n [x(.)] with appropriate coefficients a n (M ) that may depend on the Markov generator M (34) Since all the trajectories x(0 ≤ t ≤ T ) that have the same empirical observables…”

Thermodynamic bounds on correlation times

“…For a nonequilibrium system, on the other hand, the correlation time is reduced, τ z ≤ τ z eq , corresponding to faster self-averaging [11][12][13][14]. Here τ z eq is the correlation time in the equilibrium system with the same steady state.…”