Learning effective stochastic differential equations from microscopic simulations: Linking stochastic numerics to deep learning

Abstract

We identify effective stochastic differential equations (SDEs) for coarse observables of fine-grained particle- or agent-based simulations; these SDEs then provide useful coarse surrogate models of the fine scale dynamics. We approximate the drift and diffusivity functions in these effective SDEs through neural networks, which can be thought of as effective stochastic ResNets. The loss function is inspired by, and embodies, the structure of established stochastic numerical integrators (here, Euler–Maruyama and Milstein); our approximations can thus benefit from backward error analysis of these underlying numerical schemes. They also lend themselves naturally to “physics-informed” gray-box identification when approximate coarse models, such as mean field equations, are available. Existing numerical integration schemes for Langevin-type equations and for stochastic partial differential equations can also be used for training; we demonstrate this on a stochastically forced oscillator and the stochastic wave equation. Our approach does not require long trajectories, works on scattered snapshot data, and is designed to naturally handle different time steps per snapshot. We consider both the case where the coarse collective observables are known in advance, as well as the case where they must be found in a data-driven manner.

Learning effective SDEs from Brownian dynamic simulations of colloidal particles

“…Even though a trend appears in the diffusivity computed through the network the computed diffusivity is practically constant along the data and the trend is just an artifact of the fitted diffusivity through the network. This would also become evident for the null models with constant diffusivity used in Dietrich et al 14…”

“…We show how this estimation can be performed, either from the statistical definition of the terms, based on the Kramers–Moyal expansion, 12,13 or via a deep learning architecture inspired by stochastic numerical integrators. 14…”

“…The loss function used in our case (based on ref. 14) is derived from the Euler–Maruyama scheme, a numerical integration method for SDEs. The scheme for the two-dimensional case,where d B t 1 , d B t 2 are normally distributed around zero with variance h i .…”

“…, 12,13 but rather (in the spirit of the early work mentioned above) on numerical stochastic integration algorithms. 14…”

“…Our work deviates from the approaches mentioned above in three key aspects: (a) we explicitly separate the latent space construction from learning the eSDE; (b) we extend the loss function informed by numerical integration schemes from ref. 14 to allow for additional parameter dependence. Our latent space is defined through Laplace–Beltrami operator eigenfunctions, so, different from ref.…”