Robust MDPs with k-rectangular uncertainty

Shie Mannor, Ofir Mebel, Huan Xu

Research output: Contribution to journalArticlepeer-review


Markov decision processes are a common tool for modeling sequential planning problems under uncertainty. In almost all realistic situations, the system model cannot be perfectly known and must be approximated or estimated. Thus, we consider Markov decision processes under parameter uncertainty, which effectively adds a second layer of uncertainty. Most previous studies restrict to the case that uncertainties among different states are uncoupled, which leads to conservative solutions. On the other hand, robust MDPs with general coupled uncertainty sets are known to be computationally intractable. In this paper we make a first attempt at identifying subclasses of coupled uncertainty that are flexible enough to overcome conservativeness yet still lead to tractable problems. We propose a new class of uncertainty sets termed "k-rectangular uncertainty sets"-a geometric concept defined by the cardinality of possible conditional projections of the uncertainty set. The proposed scheme can model several intuitive formulations of coupled uncertainty that naturally arise in practice and leads to tractable formulations via state space augmentation.

Original languageEnglish
Pages (from-to)1484-1509
Number of pages26
JournalMathematics of Operations Research
Issue number4
StatePublished - Nov 2016


  • Dynamic Programming
  • Markov
  • Models
  • Optimal Control

All Science Journal Classification (ASJC) codes

  • Mathematics(all)
  • Computer Science Applications
  • Management Science and Operations Research


Dive into the research topics of 'Robust MDPs with k-rectangular uncertainty'. Together they form a unique fingerprint.

Cite this