Work

Causal Explanations for Sequential Decision-Making in Multi-Agent Systems

Year: 2024

Type: conference-paper

Abstract: We present CEMA: Causal Explanations in Multi-A gent systems; a framework for creating causal natural language explanations of an agent's decisions in dynamic sequential multi-agent systems to build more trustworthy autonomous agents. Unlike prior work that assumes a fixed causal structure, CEMA only requires a probabilistic model for forward-simulating the state of the system. Using such a model, CEMA simulates counterfactual worlds that identify the salient causes behind the agent's decisions. We evaluate CEMA on the task of motion planning for autonomous driving and test it in diverse simulated scenarios. We show that CEMA correctly and robustly identifies the causes behind the agent's decisions, even when a large number of other agents is present, and show via a user study that CEMA's explanations have a positive effect on participants' trust in autonomous vehicles and are rated as high as high-quality baseline explanations elicited from other participants. We release the collected explanations with annotations as the HEADD dataset.

Source: Adaptive Agents and Multi-Agents Systems

DOI: 10.65109/fhaa7375

Authors: Balint Gyevnar, Cheng Wang, Christopher G. Lucas, Shay B. Cohen, Stefano V. Albrecht

Institution: University of Edinburgh

Language: English

Topic: Explainable Artificial Intelligence (XAI)

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: UK Research and Innovation

Award: G373773752