Mixing time estimation in reversible Markov chains from a single sample path

Abstract

The spectral gap γ⋆ of a finite, ergodic, and reversible Markov chain is an important parameter measuring the asymptotic rate of convergence. In applications, the transition matrix P may be unknown, yet one sample of the chain up to a fixed time n may be observed. We consider here the problem of estimating γ⋆ from this data. Let π be the stationary distribution of P , and π⋆ = minx π(x). We show that if n =Õ 1 γ⋆π⋆ , then γ can be estimated to within multiplicative constants with high probability. When π is uniform on d states, this matches (up to logarithmic correction) a lower bound ofΩ d γ⋆ steps required for precise estimation of γ⋆. Moreover, we provide the first procedure for computing a fully data-dependent interval, from a single finitelength trajectory of the chain, that traps the mixing time t mix of the chain at a prescribed confidence level. The interval does not require the knowledge of any parameters of the chain. This stands in contrast to previous approaches, which either only provide point estimates, or require a reset mechanism, or additional prior knowledge. The interval is constructed around the relaxation time t relax = 1/γ⋆, which is strongly related to the mixing time, and the width of the interval converges to zero roughly at a 1/ √ n rate, where n is the length of the sample path.

Adaptive Huber regression on Markov-dependent data

“…Furthermore, this theorem allows γ to approach to 1 as n increases, so long as s (1 + γ)/(1 − γ) · log d/n → 0. Even though the spectral gap of the Markov chain is difficult to accurately compute in practice (Hsu, Kontorovich and Szepesvári, 2015), Theorem 1 also apply if one replaces γ with an inaccurate overestimate γ ′ ≥ γ.…”

Computationally Efficient Estimation of the Spectral Gap of a Markov Chain

“…Their method requires time O(n + |Ω| 3 ) and space O(|Ω| 2 ), and is hence mainly adapted to state spaces of moderate size. Our contribution is therefore complementary to [4], which is mainly concerned with statistical efficiency for moderate Ω while we are interested computationally efficient approaches for large Ω.…”

“…One is not allowed to choose the starting state X 0 . This is the model considered in [4]. Furthermore, since a sample path of length n can be generated by n calls to NextState, estimation in the USP model is more arduous than in the RTF model.…”

“…To get an estimation error of , UCPI requires at most n 1 = O( − 2 1−r |Ω| r+1 r−1 ) samples, time O(n 1 ) and memory O((ln n 1 ) 2 ). By comparison, the method of [4] requires at least n 2 = O(( π m (1 − λ )) −2 ) samples, with time O(n 2 + |Ω| 3 ) and memory O(|Ω| 2 ), where π m = min x∈Ω π(x). It is noted that (π m ) −1 > |Ω|, and that π m may be arbitrarily small, so that n 2 is not necessarily smaller than n 1 .…”

“…Finally, as shown by [4], in order to reach any fixed accuracy, n should satisfy n ≥ O( 1 πm ) ≥ O(|Ω|). So it is not possible to design an algorithm whose sample complexity is sub-linear in the state space size.…”