Mixing time estimation in reversible markov chains from a single sample path

Daniel Hsu, Aryeh Kontorovich, David A. Levin, Yuval Peres, Csaba Szepesvári, Geoffrey Wolfer

Research output: Contribution to journalArticlepeer-review

14 Scopus citations

Abstract

The spectral gap γ∗ of a finite, ergodic and reversible Markov chain is an important parameter measuring the asymptotic rate of convergence. In applications, the transition matrix P may be unknown, yet one sample of the chain up to a fixed time n may be observed. We consider here the problem of estimating γ∗ from this data. Let π be the stationary distribution of P, and π∗ = minx π(x). We show that if n is at least 1 / γ∗π∗ times a logarithmic correction, then γ∗ can be estimated to within a multiplicative factor with high probability. When π is uniform on d states, this nearly matches a lower bound of d / γ∗ steps required for precise estimation of γ∗. Moreover, we provide the first procedure for computing a fully data-dependent interval, from a single finite-length trajectory of the chain, that traps the mixing time tmix of the chain at a prescribed confidence level. The interval does not require the knowledge of any parameters of the chain. This stands in contrast to previous approaches, which either only provide point estimates, or require a reset mechanism, or additional prior knowledge. The interval is constructed around the relaxation time trelax = 1/γ∗, which is strongly related to the mixing time, and the width of the interval converges to zero roughly at a 1/ √ n rate, where n is the length of the sample path.

Original languageEnglish
Pages (from-to)2439-2480
Number of pages42
JournalAnnals of Applied Probability
Volume29
Issue number4
DOIs
StatePublished - 1 Jan 2019

Keywords

  • Empirical confidence interval
  • Markov chains
  • Mixing time
  • Spectral gap

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Fingerprint

Dive into the research topics of 'Mixing time estimation in reversible markov chains from a single sample path'. Together they form a unique fingerprint.

Cite this