Projected metastable Markov processes and their estimation with observable operator models

Abstract

The determination of kinetics of high-dimensional dynamical systems, such as macromolecules, polymers, or spin systems, is a difficult and generally unsolved problem - both in simulation, where the optimal reaction coordinate(s) are generally unknown and are difficult to compute, and in experimental measurements, where only specific coordinates are observable. Markov models, or Markov state models, are widely used but suffer from the fact that the dynamics on a coarsely discretized state spaced are no longer Markovian, even if the dynamics in the full phase space are. The recently proposed projected Markov models (PMMs) are a formulation that provides a description of the kinetics on a low-dimensional projection without making the Markovianity assumption. However, as yet no general way of estimating PMMs from data has been available. Here, we show that the observed dynamics of a PMM can be exactly described by an observable operator model (OOM) and derive a PMM estimator based on the OOM learning.

Markov state models from short non-equilibrium simulations—Analysis and correction of estimation bias

“…Here we only summarize a few aspects of OOMs that are relevant to the present paper and present an algorithm that can be used to estimate MSMs without bias from the initial trajectory distribution. To fully understand the theoretical background and derivation, please refer to [20][21][22].…”

“…(38)(39) that if an unknown similarity transform R ∈ R M ×M affects all OOM quantities viâ To construct an exact unbiased estimator we need three ingredients: (i) the expectation values of the empirical count matrix C τ ρ , (ii) the vector of total counts c ρ from Eqs. (20)(21), and additionally (iii) the two-step count matrices…”

“…Also, let c ρj denote the corresponding correlation from Eqs. (20)(21) and (43) w.r.t. the density ρ j .…”

“…III. OOMs are powerful finite-dimensional models that provide unbiased estimates of stationary and kinetic properties of stochastic processes under fairly mild assumptions, see [20][21][22]. Most importantly, OOMs can be estimated from non-equilibrium simulations [22] and are not limited to a local equilibrium assumption.3.…”

Exploring the free energy gain of phase separation via Markov state modeling

“…The effect of the initial distribution onto the MSM quality and the existence of this bias due to local nonequilibrium has been reported in the literature 8 and possible approaches to resolve these issues have been proposed 37,38 . In particular, these approaches consist of discarding the assumption that dynamics are Markovian on the discretized partition of the phase space, which leads to the concept of Projected Markov Models.…”