Stochastic gradient descent and fast relaxation to thermodynamic equilibrium: a stochastic control approach

Abstract

We study the convergence to equilibrium of an underdamped Langevin equation that is controlled by a linear feedback force. Specifically, we are interested in sampling the possibly multimodal invariant probability distribution of a Langevin system at small noise (or low temperature), for which the dynamics can easily get trapped inside metastable subsets of the phase space. We follow [Chen et al., J. Math. Phys. 56, 113302, 2015] and consider a Langevin equation that is simulated at a high temperature, with the control playing the role of a friction that balances the additional noise so as to restore the original invariant measure at a lower temperature. We discuss different limits as the temperature ratio goes to infinity and prove convergence to a limit dynamics. It turns out that, depending on whether the lower ("target") or the higher ("simulation") temperature is fixed, the controlled dynamics converges either to the overdamped Langevin equation or to a deterministic gradient flow. This implies that (a) the ergodic limit and the large temperature separation limit do not commute in general, and that (b) it is not possible to accelerate the speed of convergence to the ergodic limit by making the temperature separation larger and larger. We discuss the implications of these observation from the perspective of stochastic optimisation algorithms and enhanced sampling schemes in molecular dynamics.

Choice of Damping Coefficient in Langevin Dynamics

“…Remark 5. The result is consistent with related results from [4,12] that consider optimal damping coefficients that maximise the speed of convergence measured in relative entropy. Specifically, calling η t = N (µ t , Σ t ) the law of the solution to (13), with initial conditions (Q t , P t ) = (q, p); see A for details.…”

“…It is a straightforward calculation to show that the maximum value for α (that gives the fastest decay of KL(η t , ρ)) is attained at γ = 2, which is in agreement with the IAcT analysis. For analogous statements on the multidimensional case, we refer to [4]. We should mention that that there may be cases, in which the optimal damping coefficient may lead to a stiff Langevin equation, depending on the eigenvalue spectrum of the Hessian of the potential energy function.…”

“…The special case of a constant scalar diffusion coefficient is the topic of this article. The motivation for this study is a suspicion that proposed novel MCMC propagators based on Langevin dynamics (in particular, stochastic gradient methods for machine learning [4,9]) might be obtaining their advantage at the expense of reduced sampling efficiency, as, say, measured by effective sample size.…”

“…[8,17,25] Langevin dynamics has been used to tackle problems in machine learning and stochastic optimisation. From a theoretical point of view, the Langevin equation is more difficult to analyse than its overdamped counterpart, since the noise term is degenerate and the associated propagator is non-symmetric; recent work on optimising the friction coefficient for sampling is due to [11,36,4], theoretical analyses using both probabilistic and functional analytical methods have been conducted in [10,5,12]; see also [27,] and the references therein.…”