Inferring Markov chains: Bayesian estimation, model comparison, entropy rate, and out-of-class modeling

Abstract

Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer k-th order Markov chains, for arbitrary k, from finite data by applying Bayesian methods to both parameter estimation and model-order selection. Extending existing results for multinomial models of discrete data, we connect inference to statistical mechanics through information-theoretic (type theory) techniques. We establish a direct relationship between Bayesian evidence and the partition function which allows for straightforward calculation of the expectation and variance of the conditional relative entropy and the source entropy rate. Finally, we introduce a novel method that uses finite data-size scaling with model-order comparison to infer the structure of out-of-class processes.

Regional patterns in technological progress of Poland: the role of EU structural funds

“…27 In general, neither of these modifications affected the general conclusions resulting from the unmodified ML-based estimators. In the next step of the stability analysis, we went beyond the limitations of the time-homogenous Markov chains and analyzed the non-homogenous discrete-time Markov chains estimated via the maximum a posteriori method implemented using Bayesian inference (Strelioff et al 2007). This way, we obtained two sets of estimators (one for normalized capital productivity and one for normalized labor productivity), each containing the 18 annual transition matrices defined in (6).…”

Skill importance in women’s soccer

“…To specify a prior on the transition probabilities, we follow Strelioff, Crutchfield, and Hubler (2007), who approach inference for Markov chain models including parameter estimation and model comparison in a Bayesian framework. They describe the product of independent Dirichlet distributions as the conjugate prior to (4).…”

Logit Analysis of Genetic Data

“…The problem of estimating the Markov order or, more generally, the context tree has been addressed in many papers, see [8,9,14,15,17,19] and references therein. In [8] and [9] the strong consistency of the BIC estimator, without any prior bound on memory length,of the Markov order and the context tree, respectively, is demonstrated.…”