Stochastic comparative statics in Markov decision processes

Research output: Contribution to journalArticlepeer-review

Abstract

In multiperiod stochastic optimization problems, the future optimal decision is a random variable whose distribution depends on the parameters of the optimization problem. I analyze how the expected value of this random variable changes as a function of the dynamic optimization parameters in the context of Markov decision processes. I call this analysis stochastic comparative statics. I derive both comparative statics results and stochastic comparative statics results showing how the current and future optimal decisions change in response to changes in the single-period payoff function, the discount factor, the initial state of the system, and the transition probability function. I apply my results to various models from the economics and operations research literature, including investment theory, dynamic pricing models, controlled random walks, and comparisons of stationary distributions.

Original languageEnglish
Pages (from-to)797-810
Number of pages14
JournalMathematics of Operations Research
Volume46
Issue number2
DOIs
StatePublished - May 2021
Externally publishedYes

Keywords

  • Comparative statics
  • Markov decision processes
  • Stochastic comparative statics

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • General Mathematics
  • Management Science and Operations Research

Fingerprint

Dive into the research topics of 'Stochastic comparative statics in Markov decision processes'. Together they form a unique fingerprint.

Cite this