Intention-aware Policy Graphs: Explaining Agent Behavior through Desires and Commitments

The core message of this work is to propose a methodology for constructing post-hoc explainable models of agent behavior, called Intention-aware Policy Graphs (IPGs), that can answer questions about an agent's intentions, plans, and reasons for actions, even when the agent's internal model is opaque.