Gambling Teams and Waiting Times for Patterns in Two-State Markov Chains

Abstract

Methods using gambling teams and martingales are developed and applied to find formulae for the expected value and the generating function of the waiting time to observation of an element of a finite collection of patterns in a sequence generated by a two-state Markov chain of first, or higher, order.

Stopping Probabilities for Patterns in Markov Chains

“…Several distinct techniques have been used to solve these problems for both independent or Markov-dependent trials; see [1], [2], [4], [5], [6], [7], [8], and the references therein. We are interested in computing the stopping probabilities P(τ = τ A q ), q = 1, .…”

“…, K. (1.2) In order to well define (1.1) and avoid ties, we assume that no pattern from C contains another pattern as a subpattern. We shall show that the martingale methods introduced in [5] and [8], and further developed in [6], [9], and [10] may also be applied to compute stopping probabilities of a sequence of patterns for finite-state Markov chains. Feller [3] studied the occurrence of patterns in independent Bernoulli trials using recurrent event theory.…”

“…In [11] run probabilities were computed as a function of the number n of trials for a collection of runs, but not as a function of the variable in (1.1) or (1.2). In this paper we bring together the methods developed for gambling teams in [6] and [9] to obtain the probabilities in (1.2). First, in Section 2 we use the procedure developed in [6] to get (1.2) for a two-state Markov chain.…”

On occurrence of patterns in Markov chains: Method of gambling teams

“…But as we have shown in our recent paper that is not exactly true. More specifically, in Glaz et al (2006) the martingale technique was developed to treat occurrence of patterns in two-state Markov chains. Here we present another algorithm based on the method of gambling teams that works for multi-state Markov chains as well.…”

“…Here we present another algorithm based on the method of gambling teams that works for multi-state Markov chains as well. Some ideas and devices in this article are similar to those in Glaz et al (2006) but the overall technique is substantially different even when it is applied to two-state Markov chains.…”