On occurrence of patterns in Markov chains: Method of gambling teams

Abstract

a b s t r a c tA new martingale technique is developed to find formulas for the first two moments and generating function of the waiting time until one observes an element of a finite collection of patterns in a finite multi-state Markov chain.

Stopping Probabilities for Patterns in Markov Chains

“…Remarks on these assumptions can be found in Section 2 of [9]. We just add the remark that assumptions 1 and 2 together ensure that q P(τ = τ A q ) = 1.…”

“…We now use the method of gambling teams given in [9]. Before continuing, we suggest that the reader view the details of Section 3 of [9] to understand what follows.…”

“…, K. (1.2) In order to well define (1.1) and avoid ties, we assume that no pattern from C contains another pattern as a subpattern. We shall show that the martingale methods introduced in [5] and [8], and further developed in [6], [9], and [10] may also be applied to compute stopping probabilities of a sequence of patterns for finite-state Markov chains. Feller [3] studied the occurrence of patterns in independent Bernoulli trials using recurrent event theory.…”

“…In [11] run probabilities were computed as a function of the number n of trials for a collection of runs, but not as a function of the variable in (1.1) or (1.2). In this paper we bring together the methods developed for gambling teams in [6] and [9] to obtain the probabilities in (1.2). First, in Section 2 we use the procedure developed in [6] to get (1.2) for a two-state Markov chain.…”

“…First, in Section 2 we use the procedure developed in [6] to get (1.2) for a two-state Markov chain. In Section 3 we apply the algorithm stated in [9] to get (1.2) for a multistate Markov chain. Both methods and their results are similar, but even when we consider only two-state Markov chains, these methods show us that the way we calculate the probabilities in (1.2) are different.…”