Maximum Entropy Rate Reconstruction of Markov Dynamics

Abstract

We develop ideas proposed by Van der Straeten to extend maximum entropy principles to Markov chains. We focus in particular on the convergence of such estimates in order to explain how our approach makes possible the estimation of transition probabilities when only short samples are available, which opens the way to applications to non-stationary processes. The current work complements an earlier communication by providing numerical details, as well as a full derivation of the multi-constraint two-state and three-state maximum entropy transition matrices.

Information Entropy Production of Maximum Entropy Markov Chains from Spike Trains

“…We have focused on spike train statistics, but our results are not restricted to this field and can be applied wherever Markov maximum entropy measures under constraints have to be inferred from data, especially for irreversible Markov chains arising from stochastic network theory [ 49 ], information theory [ 37 ], and finance [ 38 ], among other disciplines.…”

“…After that, one maximizes the information entropy rate, which is a concave functional in the space of Lagrange multipliers associated to the constraints, obtaining the unique Markov measure that better approximates the statistics among all probability measures that match exactly the constraints [ 23 ]. To to our knowledge, previous approaches ignore how to deal with the inference of irreversible Markov processes in the maximum entropy context [ 37 , 38 ].…”

Large Deviations Properties of Maximum Entropy Markov Chains from Spike Trains

“…However, the Gärtner-Ellis theorem provides a smart shortcut for avoiding this problem [14]. To this end, let us introduce the scaled cumulant generating function (SCGF) 6 associated to the random variable f , by λ f (k) =: lim…”

Large Deviations Properties of Maximum Entropy Markov Chains from Spike Trains

“…Recall that the goal is no longer to estimate a probability distribution, but to reconstruct from data a transition matrix P and a corresponding invariant measure . On the one hand, the challenge is that as P and are not independent parameters of the process ( has to be the eigenvector associated with the unitary eigenvalue of P [ 48 ]), and, on the other hand, although Lagrange multipliers method can still be applied for constraints of range two or more, the extension for non-synchronous constraints is not straightforward. For this reason, in the sequel, we explore an alternative route to build the MEMC based on the transfer matrix technique.…”