Usual and stochastic tail orders between hitting times for two Markov chains

Abstract

We consider time-homogeneous Markov chains with state space E k ≡ {0, 1, … , k} and initial distribution concentrated on the state 0. For pairs of such Markov chains, we study the Stochastic Tail Order and the stochastic order in the usual sense between the respective first passage times in the state k. On this purpose, we will develop a method based on a specific relation between two stochastic matrices on the state space E k . Our method provides comparisons that are simpler and more refined than those obtained by the analysis based on the spectral gaps.

Stochastic Precedence and Minima Among Dependent Variables

“…Proof. The implication b) ⇒ a) is immediate in view of the identity (11). The implication a) ⇒ b) is proved by contradiction.…”

“…This field is also related to the analysis of stochastic comparisons for hitting times for Markov chains see e.g. [10,11] and references therein.…”

Reachability and Safety Objectives in Markov Decision Processes on Long but Finite Horizons

“…Let σ be a strategy as in Theorem 5.1, σ = σ be a stationary strategy and s be the initial state. One can compute a horizon T such that σ outperforms σ beyond T , i.e., P sσ (t * ≤ t) > P sσ (t * ≤ t) for all t ≥ T , by using inequalities (A1) and (A2) in [21].…”

Ranking graphs through hitting times of Markov chains

“…From a theoretical point of view, we emphasize that the structure of the hitting times is very particular and many properties on their distributions are known a priori, see for example [13, 18] and references therein. Moreover, the stationary Markov chains used in our construction have the particular property that the square of the transition matrix has all of its entries equal.…”