Estimating the transition matrix of a Markov chain observed at random times

Abstract

ABSTRACT. In this paper we develop a statistical estimation technique to recover the transition kernel P of a Markov chain X = (X m ) m∈N in presence of censored data. We consider the situation where only a sub-sequence of X is available and the time gaps between the observations are iid random variables. Under the assumption that neither the time gaps nor their distribution are known, we provide an estimation method which applies when some transitions in the initial Markov chain X are known to be unfeasible. A consistent estimator of P is derived in closed form as a solution of a minimization problem. The asymptotic performance of the estimator is then discussed in theory and through numerical simulations.

Relations Between Connected and Self-Avoiding Hikes in Labelled Complete Digraphs

“…The construction gives in this case a non-trivial sparse matrix in the commutant of W. This kind of problems has some applications in practice, for instance in the study of random processes. In [2], the authors investigate conditions under which the transition kernel of a finite state Markov chain observed at random times can be estimated consistently. They show that sparsity conditions, arising from particular state transitions known to be impossible, suffice to recover the transition kernel when it commutes with a certain matrix for which an estimator is available.…”

Hypothesis testing for Markovian models with random time observations

“…d) More generally, if one or several state transitions are known to be impossible in the initial chain (X m ) m∈N , the nullity of the corresponding entries can be expressed as a set of linear conditions on p. The number of constraints is equal to the number of known zeros in P . This situation is treated in [3]. e) A symmetric transition matrix P corresponds to N (N − 1)/2 linear constraints on p = vec(P ).…”

“…Thus, given the model M, the possible values for p can be restricted to M ∩ Com(Q), where Com(Q) is the commutant of Q. As in [3], we consider the commutation…”

“…From Lemma 6.1 in [3], this condition is actually equivalent to ∆(Q)Φ 0 being of full rank, in which case…”

“…As explained in [3], a typical application of this setting occurs when considering a continuous time Markov chain Z = (Z t ) t>0 observed at a discrete time grid t 1 < ... < t n . In this situation, X represents the jump process of Z and (τ k ) k∈N denotes the number of jumps occurring between two consecutive observations, Y k := Z t k .…”