Domain independent heuristics for online stochastic contingent planning

Oded Blumenthal, Guy Shani

Research output: Contribution to journalArticlepeer-review

Abstract

Partially observable Markov decision processes (POMDP) are a useful model for decision-making under partial observability and stochastic actions. Partially Observable Monte-Carlo Planning (POMCP) is an online algorithm for deciding on the next action to perform, using a Monte-Carlo tree search approach, based on the UCT algorithm for fully observable Markov-decision processes. POMCP develops an action-observation tree, and at the leaves, uses a rollout policy to provide a value estimate for the leaf. As such, POMCP is highly dependent on the rollout policy to compute good estimates, and hence identify good actions. Thus, many practitioners who use POMCP are required to create strong, domain-specific heuristics. In this paper, we model POMDPs as stochastic contingent planning problems. This allows us to leverage domain-independent heuristics that were developed in the planning community. We suggest two heuristics, the first is based on the well-known hadd heuristic from classical planning, and the second is computed in belief space, taking the value of information into account.

Original languageAmerican English
JournalAnnals of Mathematics and Artificial Intelligence
DOIs
StateAccepted/In press - 1 Jan 2024

Keywords

  • Contingent planning
  • Heuristics
  • Online planning
  • POMDP

All Science Journal Classification (ASJC) codes

  • Applied Mathematics
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Domain independent heuristics for online stochastic contingent planning'. Together they form a unique fingerprint.

Cite this