Adapting to Mixing Time in Stochastic Optimization with Markovian Data

Ron Dorfman, Kfir Yehuda Levy

Research output: Contribution to journalConference articlepeer-review

Abstract

We consider stochastic optimization problems
where data is drawn from a Markov chain. Existing methods for this setting crucially rely on
knowing the mixing time of the chain, which in
real-world applications is usually unknown. We
propose the first optimization method that does
not require the knowledge of the mixing time, yet
obtains the optimal asymptotic convergence rate
when applied to convex problems. We further
show that our approach can be extended to: (i)
finding stationary points in non-convex optimization with Markovian data, and (ii) obtaining better
dependence on the mixing time in temporal difference (TD) learning; in both cases, our method
is completely oblivious to the mixing time. Our
method relies on a novel combination of multilevel Monte Carlo (MLMC) gradient estimation
together with an adaptive learning method.
Original languageEnglish
Pages (from-to)5429-5446
JournalProceedings of Machine Learning Research
Volume162
StatePublished - 2022
EventProceedings of the 39th International Conference on Machine Learning - Baltimore, United States
Duration: 17 Jul 202223 Jul 2022
https://proceedings.mlr.press/v162/

Fingerprint

Dive into the research topics of 'Adapting to Mixing Time in Stochastic Optimization with Markovian Data'. Together they form a unique fingerprint.

Cite this