Estimation of the Entropy Rate of a Countable Markov Chain

Estimation using plug-in of the stationary distribution and Shannon entropy of continuous time Markov processes

“…See also Girardin and Sesboüé (2009) for a detailed study of such estimators for two-state Markov chains. For ergodic continuous-time processes, Regnault (2009) established the good asymptotic behavior of plug-in estimators of the entropy of the stationary distribution of a two-state continuous-time ergodic Markov process, by extending the methods developed by Ciuperca and Girardin (2007) to continuous-time processes.…”

“…Due to the few assumptions made on the sequences, the behavior of the estimators remains partly unknown. Ciuperca and Girardin (2007) computed plug-in estimators of the entropy of the stationary distribution of a finite-state ergodic Markov chain. For such a sequence, the stationary distribution is an explicit function of the transition probability distributions.…”

Different Methods of Partitioning the Phase Space of a Dynamic System

“…An entropy rate approach has been adopted for selecting the alphabet size [23]. Let h(M (k) ) denote the entropy rate of the transition matrix for iteration k [4].…”

Measuring the quality of life through Markov reward processes: Analysis and inference

“…In particular, the determination of the asymptotic variances of the estimators allows us to construct the confidence set of the rewards for each significance level. The goal is achieved developing the techniques of estimation for Markov chains, presented in Billingsley (1960), in Sadek and Limnios (2002), in Limnios et al (2004) and Ciuperca and Girardin (2007) together with results in nonparametric empirical estimation. Gardiner et al (2006) and Gardiner et al (2008) considered estimators of the first moment of a nonhomogeneous Markov reward process with deterministic reward functions.…”