Toward robust policy summarization

Isaac Lage, Finale Doshi-Velez, Daphna Lifschitz, Ofra Amir

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

AI agents are being developed to help people with high stakes decision-making processes from driving cars to prescribing drugs. It is therefore becoming increasingly important to develop "explainable AI" methods that help people understand the behavior of such agents. Summaries of agent policies can help human users anticipate agent behavior and facilitate more effective collaboration. Prior work has framed agent summarization as a machine teaching problem where examples of agent behavior are chosen to maximize reconstruction quality under the assumption that people do inverse reinforcement learning to infer an agent's policy from demonstrations. We compare summaries generated under this assumption to summaries generated under the assumption that people use imitation learning. We show through simulations that in some domains, there exist summaries that produce high-quality reconstructions under different models, but in other domains, only matching the summary extraction model to the reconstruction model produces high-quality reconstructions. These results highlight the importance of assuming correct computational models for how humans extrapolate from a summary, suggesting human-in-the-loop approaches to summary extraction.

Original languageAmerican English
Title of host publication18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019
Pages2081-2083
Number of pages3
ISBN (Electronic)9781510892002
StatePublished - 1 Jan 2019
Event18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019 - Montreal, Canada
Duration: 13 May 201917 May 2019
https://dl.acm.org/doi/proceedings/10.5555/3306127

Publication series

NameProceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
Volume4

Conference

Conference18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019
Country/TerritoryCanada
CityMontreal
Period13/05/1917/05/19
Internet address

Keywords

  • Explainable AI
  • Policy summarization

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering

Cite this