On the Approximation Quality of Markov State Models

Abstract

We consider a continuous-time Markov process on a large continuous or discrete state space. The process is assumed to have strong enough ergodicity properties and to exhibit a number of metastable sets. Markov state models (MSMs) are designed to represent the effective dynamics of such a process by a Markov chain that jumps between the metastable sets with the transition rates of the original process. MSMs have been used for a number of applications, including molecular dynamics, for more than a decade. Their approximation quality, however, has not yet been fully understood. In particular, it would be desirable to have a sharp error bound for the difference in propagation of probability densities between the MSM and the original process on long timescales. Here, we provide such a bound for a rather general class of Markov processes ranging from diffusions in energy landscapes to Markov jump processes on large discrete spaces. Furthermore, we discuss how this result provides formal support or shows the limitations of algorithmic strategies that have been found to be useful for the construction of MSMs. Our findings are illustrated by numerical experiments.

Optimal control of molecular dynamics using Markov state models

“…Recently the interest in MSMs has increased a lot, for it had been demonstrated that MSMs can be constructed even for very high dimensional systems [25]. They have been especially useful for modelling the interesting slow dynamics of biomolecules [21,[28][29][30][31][32] and materials [33] (there under the name "kinetic Monte Carlo"). If the system exhibits metastability and the jump process between the metastable sets are approximately Markovian, the corresponding MSM simply describes the Markov process that jumps between the sets with the aggregated statistics of the original process.…”

Estimation and uncertainty of reversible Markov models

“…24,31,32 The error incurred by the discretization and by the subsequent approximation of the jump-process as a Markov process can be systematically controlled and evaluated. 7,[33][34][35] a) B. Trendelkamp-Schroer and H. Wu contributed equally to this work. b) Electronic mail: frank.noe@fu-berlin.…”

“…The parameter τ is called lag time and is crucial for the quality of the Markov model. 7,33 Next, one computes the transition matrix either by maximizing the likelihood, i.e., the probability over all possible Markov model transition (or rate) matrices that may have generated the observed transition counts 7,36,37 or by sampling Markov models from the posterior distribution. [38][39][40][41][42] A maximum likelihood estimate gives a single-point estimate, i.e., a single Markov model that is "most representative" given the data.…”

Diffusion maps, clustering and fuzzy Markov modeling in peptide folding transitions

“…This approach has a long tradition in chemistry, and, in particular, in the concise description of population changes in chemical reaction kinetics. Such projections onto discrete sets of states naturally lead to descriptions of the dynamics in terms of Markov state models 14 or coarse master equations 15,16 with discrete timestepping or continuous dynamics, respectively. Coarse master equations and Markov state models have attracted much attention because they can be constructed directly from molecular dynamics simulations, [15][16][17][18][19][20][21] with the aim to capture the dynamics of most interest, occurring over long time scales.…”