Approximate Optimal Controls via Instanton Expansion for Low Temperature Free Energy Computation

Abstract

The computation of free energies is a common issue in statistical physics. A natural technique to compute such high-dimensional integrals is to resort to Monte Carlo simulations. However, these techniques generally suffer from a high variance in the low temperature regime, because the expectation is often dominated by high values corresponding to rare system trajectories. A standard way to reduce the variance of the estimator is to modify the drift of the dynamics with a control enhancing the probability of rare events, leading to so-called importance sampling estimators. In theory, the optimal control leads to a zero-variance estimator; it is, however, defined implicitly and computing it is of the same difficulty as the original problem. We propose here a general strategy to build approximate optimal controls in the small temperature limit for diffusion processes, with the first goal to reduce the variance of free energy Monte Carlo estimators. Our construction builds upon low noise asymptotics by expanding the optimal control around the instanton, which is the path describing most likely fluctuations at low temperature. This technique not only helps reducing variance, but it is also interesting as a theoretical tool since it differs from usual small temperature expansions (WKB ansatz). As a complementary consequence of our expansion, we provide a perturbative formula for computing the free energy in the small temperature regime, which refines the now standard Freidlin--Wentzell asymptotics. We compute this expansion explicitly for lower orders, and explain how our strategy can be extended to an arbitrary order of accuracy. We support our findings with illustrative numerical examples.

Symmetries and Zero Modes in Sample Path Large Deviations

“…Importantly, one obtains the full limiting rare event probability or probability density instead of merely its exponential scaling, in regimes where direct sampling methods are completely intractable. In this paper, we have first set out to rederive such prefactor formulas at leading order for unique instantons [30][31][32][33], expressed in terms of Riccati matrices, explicitly using tools from field theory, i.e. by evaluating the appearing functional determinants using Forman's theorem [40].…”

“…Example B. 4 Since previous papers [30][31][32][33] have mostly dealt with additive noise, we test the more general case of multiplicative noise that is included here in a simple toy example. Consider the one-dimensional Itô SDE…”

“…As an additional, concrete motivation to go beyond such rough estimates in practical applications, it has been pointed out very recently that for assessing the relative importance of different instantonic transition paths, knowledge of the LDT prefactor at leading order may be vital even at comparably small noise strengths [29]. In the last year, there has been a lot of activity to provide generic numerical tools that also allow for the computation of the leading order term of the large deviation prefactor for the statistics of final time observables of small noise ordinary SDEs using symmetric Riccati matrix differential equations, either forward or backward in time [30][31][32][33]. In an abstract setting, expressions for prefactors in this context, even at arbitrarily high order, have already been known rigorously since the 1980's [2,[34][35][36] and are, not surprisingly, related to a certain operator determinant at the leading order.…”

“…Relevant examples of this phenomenon in the context of sample path LDT include the one-dimensional Kardar-Parisi-Zhang (KPZ) equation [43][44][45][46] for the surface height at one point in space and with two-sided Brownian motion initial condition (leading to discrete mirror symmetry breaking), the two-dimensional [47] and three-dimensional [48] incompressible Navier-Stokes equations and a Lagrangian turbulence model [49] (all with rotational symmetry breaking). In all of these cases, due to the underlying symmetries, it turns out that it suffices to integrate a single Riccati equation, corresponding to a single reduced functional determinant evaluation, which thereby allows for a generalization of earlier results [30][31][32][33] without increasing the computational costs. In addition to the examples listed above, further systems where the methods and results of this paper could be applied are those within the scope of the macroscopic fluctuation theory [50], e.g.…”

Stochastic viscosity approximations of Hamilton–Jacobi equations and variance reduction

“…We prove, under geometric conditions, that the estimator proposed in this work is a stochastic viscosity approximation of order 1, hence it is 1-log efficient. We then turn to the next order approximation presented in [12], showing that under similar conditions it provides a stochastic viscosity approximation of order 2, hence it is 2-log efficient and with logarithmic relative variance decaying linearly to zero. A simple numerical application confirms our theoretical findings.…”