Monte Carlo Planning in Hybrid Belief POMDPs

Moran Barenboim, Moshe Shienman, Vadim Indelman

Research output: Contribution to journalArticlepeer-review


Real-world problems often require reasoning about hybrid beliefs, over both discrete and continuous random variables. Yet, such a setting has hardly been investigated in the context of planning. Moreover, existing online partially observable Markov decision processes (POMDPs) solvers do not support hybrid beliefs directly. In particular, these solvers do not address the added computational burden due to an increasing number of hypotheses with the planning horizon, which can grow exponentially. As part of this work, we present a novel algorithm, Hybrid Belief Monte Carlo Planning (HB-MCP) that utilizes the Monte Carlo Tree Search (MCTS) algorithm to solve a POMDP while maintaining a hybrid belief. We illustrate how the upper confidence bound (UCB) exploration bonus can be leveraged to guide the growth of hypotheses trees alongside the belief trees. We then evaluate our approach in highly aliased simulated environments where unresolved data association leads to multi-modal belief hypotheses.

Original languageEnglish
Pages (from-to)4410-4417
Number of pages8
JournalIEEE Robotics and Automation Letters
Issue number8
StatePublished - 1 Aug 2023


  • Planning under uncertainty
  • autonomous agents

All Science Journal Classification (ASJC) codes

  • Mechanical Engineering
  • Control and Optimization
  • Artificial Intelligence
  • Human-Computer Interaction
  • Control and Systems Engineering
  • Computer Vision and Pattern Recognition
  • Biomedical Engineering
  • Computer Science Applications


Dive into the research topics of 'Monte Carlo Planning in Hybrid Belief POMDPs'. Together they form a unique fingerprint.

Cite this