Stopping Probabilities for Patterns in Markov Chains

Abstract

Consider a sequence of Markov-dependent trials where each trial produces a letter of a finite alphabet. Given a collection of patterns, we look at this sequence until one of these patterns appears as a run. We show how the method of gambling teams can be employed to compute the probability that a given pattern is the first pattern to occur.

Waiting times and stopping probabilities for patterns in Markov chains

“…Taking z = 1 gives f A = f B = 1/10, f C = 8/10, F 1 = F 2 = 44/15, F 3 = 24/15, and hence E(τ ) = 1 + F 1 + F 2 + F 3 = 127/15. These results are all in agreement with that in [12] and [4].…”

“…(3) To obtain E(τ ) and P (τ = τ A ) with A ∈ C, we only need to solve one linear system involving |∆| + |C| equations and |∆| + |C| variables. Compared with the results in [6], [12] and [4], it is a much easy and effective way.…”

“…We begin with the analysis of Example 1 of [12]. The mean waiting time and the generating function of τ are calculated in Example 1 and Example 3 of [12] respectively, while the stopping probability is obtained in Example 3.1 of [4]. We now recalculate all these values by applying our results.…”

“…[6], V. Pozdnyakov [12] introduced gambling teams and used Martingale theory to study E(τ ). In 2014, R. J. Gava and D. Salotti [4] obtained the system of linear equations of P (τ = τ A ) with A ∈ C based on the results of [6] and [12].…”

“…When {Z n } is a Markov chain, though the mean waiting time E(τ ) and the stopping probabilities P (τ = τ A ) were obtained in [6], [12] and [4], the method is complicated.…”