Minimax Learning of Ergodic Markov Chains

Geoffrey Wolfer; Aryeh Kontorovich

Minimax Learning of Ergodic Markov Chains

Geoffrey Wolfer, Aryeh Kontorovich

Department of Computer Science

Ben-Gurion University of the Negev

Research output: Contribution to journal › Conference article › peer-review

Abstract

We compute the finite-sample minimax (modulo logarithmic factors) sample complexity of learning the parameters of a finite Markov chain from a single long sequence of states. Our error metric is a natural variant of total variation. The sample complexity necessarily depends on the spectral gap and minimal stationary probability of the unknown chain, for which there are known finite-sample estimators with fully empirical confidence intervals. To our knowledge, this is the first PAC-type result with nearly matching (up to logarithmic factors) upper and lower bounds for learning, in any metric, in the context of Markov chains.

Original language	American English
Pages (from-to)	904-930
Number of pages	27
Journal	Proceedings of Machine Learning Research
Volume	98
State	Published - 1 Jan 2019
Event	30th International Conference on Algorithmic Learning Theory, ALT 2019 - Chicago, United States Duration: 22 Mar 2019 → 24 Mar 2019 https://proceedings.mlr.press/v98

Keywords

ergodic Markov chain
learning
minimax

All Science Journal Classification (ASJC) codes

Artificial Intelligence
Software
Control and Systems Engineering
Statistics and Probability

Cite this

@article{12846e8e4e53404eb62a4cb22dc8fd1f,

title = "Minimax Learning of Ergodic Markov Chains",

abstract = "We compute the finite-sample minimax (modulo logarithmic factors) sample complexity of learning the parameters of a finite Markov chain from a single long sequence of states. Our error metric is a natural variant of total variation. The sample complexity necessarily depends on the spectral gap and minimal stationary probability of the unknown chain, for which there are known finite-sample estimators with fully empirical confidence intervals. To our knowledge, this is the first PAC-type result with nearly matching (up to logarithmic factors) upper and lower bounds for learning, in any metric, in the context of Markov chains.",

keywords = "ergodic Markov chain, learning, minimax",

author = "Geoffrey Wolfer and Aryeh Kontorovich",

note = "Publisher Copyright: {\textcopyright} 2019 Proceedings of Machine Learning Research. All rights reserved.; 30th International Conference on Algorithmic Learning Theory, ALT 2019 ; Conference date: 22-03-2019 Through 24-03-2019",

year = "2019",

month = jan,

day = "1",

language = "American English",

volume = "98",

pages = "904--930",

journal = "Proceedings of Machine Learning Research",

issn = "2640-3498",

publisher = "ML Research Press",

url = "https://proceedings.mlr.press/v98",

}

TY - JOUR

T1 - Minimax Learning of Ergodic Markov Chains

AU - Wolfer, Geoffrey

AU - Kontorovich, Aryeh

PY - 2019/1/1

Y1 - 2019/1/1

N2 - We compute the finite-sample minimax (modulo logarithmic factors) sample complexity of learning the parameters of a finite Markov chain from a single long sequence of states. Our error metric is a natural variant of total variation. The sample complexity necessarily depends on the spectral gap and minimal stationary probability of the unknown chain, for which there are known finite-sample estimators with fully empirical confidence intervals. To our knowledge, this is the first PAC-type result with nearly matching (up to logarithmic factors) upper and lower bounds for learning, in any metric, in the context of Markov chains.

AB - We compute the finite-sample minimax (modulo logarithmic factors) sample complexity of learning the parameters of a finite Markov chain from a single long sequence of states. Our error metric is a natural variant of total variation. The sample complexity necessarily depends on the spectral gap and minimal stationary probability of the unknown chain, for which there are known finite-sample estimators with fully empirical confidence intervals. To our knowledge, this is the first PAC-type result with nearly matching (up to logarithmic factors) upper and lower bounds for learning, in any metric, in the context of Markov chains.

KW - ergodic Markov chain

KW - learning

KW - minimax

UR - http://www.scopus.com/inward/record.url?scp=85161105399&partnerID=8YFLogxK

M3 - Conference article

SN - 2640-3498

VL - 98

SP - 904

EP - 930

JO - Proceedings of Machine Learning Research

JF - Proceedings of Machine Learning Research

T2 - 30th International Conference on Algorithmic Learning Theory, ALT 2019

Y2 - 22 March 2019 through 24 March 2019

ER -

Minimax Learning of Ergodic Markov Chains

Abstract

Keywords

All Science Journal Classification (ASJC) codes

Other files and links

Fingerprint

Cite this