Abstract
AI agents are being developed to help people with high stakes decision-making processes from driving cars to prescribing drugs. It is therefore becoming increasingly important to develop "explainable AI" methods that help people understand the behavior of such agents. Summaries of agent policies can help human users anticipate agent behavior and facilitate more effective collaboration. Prior work has framed agent summarization as a machine teaching problem where examples of agent behavior are chosen to maximize reconstruction quality under the assumption that people do inverse reinforcement learning to infer an agent's policy from demonstrations. We compare summaries generated under this assumption to summaries generated under the assumption that people use imitation learning. We show through simulations that in some domains, there exist summaries that produce high-quality reconstructions under different models, but in other domains, only matching the summary extraction model to the reconstruction model produces high-quality reconstructions. These results highlight the importance of assuming correct computational models for how humans extrapolate from a summary, suggesting human-in-the-loop approaches to summary extraction.
Original language | American English |
---|---|
Title of host publication | 18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019 |
Pages | 2081-2083 |
Number of pages | 3 |
ISBN (Electronic) | 9781510892002 |
State | Published - 1 Jan 2019 |
Event | 18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019 - Montreal, Canada Duration: 13 May 2019 → 17 May 2019 https://dl.acm.org/doi/proceedings/10.5555/3306127 |
Publication series
Name | Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS |
---|---|
Volume | 4 |
Conference
Conference | 18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019 |
---|---|
Country/Territory | Canada |
City | Montreal |
Period | 13/05/19 → 17/05/19 |
Internet address |
Keywords
- Explainable AI
- Policy summarization
All Science Journal Classification (ASJC) codes
- Artificial Intelligence
- Software
- Control and Systems Engineering