Learning and identity testing of Markov chains

Geoffrey Wolfer, Aryeh Kontorovich

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review


In this chapter, we provide an overview of the two inference problems of learning and identity testing of a Markov chain based on a single trajectory of observations started from an arbitrary state. The learning problem is concerned with the estimation of the state transition probabilities of the process, while the testing problem deals with determining whether the unknown Markov chain is identical to or far from a given reference chain. We analyze both tasks from within the minimax framework and with respect to several competing notions of distance. We observe that the sample complexities depend on the number of states and often also on the stationary and mixing properties of the Markov chains. We further proceed to compare advantages and drawbacks of the different contrast functions we consider.

Original languageEnglish
Title of host publicationArtificial Intelligence
EditorsSteven G. Krantz, Arni S.R. Srinivasa Rao, C.R. Rao
PublisherElsevier B.V.
Number of pages18
ISBN (Print)9780443137631
StatePublished - 1 Jan 2023

Publication series

NameHandbook of Statistics
ISSN (Print)0169-7161


  • Ergodic Markov chain
  • Identity testing
  • Single trajectory model
  • Statistical estimation

ASJC Scopus subject areas

  • Statistics and Probability
  • Modeling and Simulation
  • Applied Mathematics


Dive into the research topics of 'Learning and identity testing of Markov chains'. Together they form a unique fingerprint.

Cite this