A relative entropy rate method for path space sensitivity analysis of stationary complex stochastic dynamics

Abstract

We propose a new sensitivity analysis methodology for complex stochastic dynamics based on the Relative Entropy Rate. The method becomes computationally feasible at the stationary regime of the process and involves the calculation of suitable observables in path space for the Relative Entropy Rate and the corresponding Fisher Information Matrix. The stationary regime is crucial for stochastic dynamics and here allows us to address the sensitivity analysis of complex systems, including examples of processes with complex landscapes that exhibit metastability, non-reversible systems from a statistical mechanics perspective, and high-dimensional, spatially distributed models. All these systems exhibit, typically non-gaussian stationary probability distributions, while in the case of high-dimensionality, histograms are impossible to construct directly. Our proposed methods bypass these challenges relying on the direct Monte Carlo simulation of rigorously derived observables for the Relative Entropy Rate and Fisher Information in path space rather than on the stationary probability distribution itself. We demonstrate the capabilities of the proposed methodology by focusing here on two classes of problems: (a) Langevin particle systems with either reversible (gradient) or non-reversible (non-gradient) forcing, highlighting the ability of the method to carry out sensitivity analysis in non-equilibrium systems; and, (b) spatially extended Kinetic Monte Carlo models, showing that the method can handle high-dimensional problems.

Analysis of Trajectory Entropy for Continuous Stochastic Processes at Equilibrium

“…A similar expression can in fact be obtained for the trajectory probability density of Markovian processes in general, 22 and the derivation for the continuous time and continuous space Langevin dynamics is shown here for completion. The second term in eq 7 can be viewed as the entropy of the conditional probability density of time propagation and is defined as the Caliber by Jaynes.…”

“…The probability density of a trajectory X(t) following the Langevin equation is described by the OM action: 29−31 (22) In this equation, the dynamic parameters of reference dynamics are labeled with the subscript of "ref" to explicitly illustrate the term cancelation discussed later. Furthermore, we also employ the ⟨g[X(t)]⟩ X(t) = ∫ X(t) [X(t)]g[X(t)] notation for an arbitrary functional of the trajectory X(t), g[X(t)].…”

Information-theoretic tools for parametrized coarse-graining of non-equilibrium extended systems

“…A similar quantity, in the context of sensitivity analysis, was recently considered in 33 , where the authors also developed efficient statistical estimators for the derivatives of RER ∂ θ k H(P | Q θ ) and ∂ 2 θiθj H(P | Q θ ). We discuss related estimators in Section IV.…”

“…While the evaluation of the Hessian Hess(H(P | Q θ n )) presents an additional computational cost, it also offers additional information about the parametrization, sensitivity and identifiability of the approximating model, 33 . Indeed the first and the second derivatives of the rate function H(P | Q θ n ) are of the form:…”

“…Furthermore, RER and in particular the pathspace FIM where introduced recently as gradient-free sensitivity analysis tools for complex, non-equilibrium stochastic systems with a wide range of applicability to reaction networks, lattice Kinetic Monte Carlo and Langevin dynamics 33 . Unlike the coarse-graining setting we propose here and where models are set in different state spaces depending on their granularity, in the latter work the same state space is assumed between compared models.…”