Work

Evaluating the effectiveness of LLMs for explainable deep reinforcement learning

Year: 2025

Type: article

Abstract: Understanding the decision-making of reinforcement learning (RL) agents is essential for real-world deployment. Existing eXplainable RL (XRL) techniques, such as feature attribution and policy visualization, provide insight but remain inaccessible to non-experts. Large Language Models (LLMs) offer a natural-language alternative, yet often lack logical consistency and alignment with agent goals. This study benchmarks three explanation generation methods: Chain-of-Thought (CoT) prompting as the standard baseline used in prior work, Monte Carlo Tree Search (MCTS) augmentation, and supervised fine-tuning (SFT) across various models. Evaluations using Soundness and Fidelity show that CoT frequently produces reasoning errors, whereas MCTS improves quality for larger models (avg. + 23 % Soundness, + 17 % Fidelity), while SFT yields greater and more consistent gains for smaller ones ( + 58 % Soundness, + 52 % Fidelity), underscoring the need to align methods with model capacity. An LLM-as-a-Judge framework further validates these findings, showing strong agreement with human assessments (weighted Cohen’s κ = 0 . 77 , Spearman ρ = 0 . 88 ), supporting scalable and reliable assessment of textual explanations.

Source: Machine Learning with Applications

DOI: 10.1016/j.mlwa.2025.100795

Authors: Ayoub Belouadah, Marcelo Luis Ruiz-Rodríguez, Sylvain Kubler, Yves Le Traon

Institutions: University of Luxembourg, Interdisciplinary Centre for Security, Reliability and Trust

Language: English

Topic: Explainable Artificial Intelligence (XAI)

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,340

Funder: Fonds National de la Recherche Luxembourg