Interpretable Meta-Reinforcement Learning : Explicit Task Inference and Epistemic Exploration in Procedural Environments
Year: 2026
Type: dissertation
Abstract: Deep Reinforcement Learning (DRL) struggles to generalise across varying environments, often memorising trajectories rather than inferring underlying causal rules. Meta-Reinforcement Learning (Meta-RL) addresses this by treating adaptation as an optimisation problem, yet standard recurrent agents act as opaque black boxes. This opacity makes it difficult to distinguish Bayes-adaptive epistemic exploration from blind statistical heuristics. This thesis introduces an interpretable Meta-RL architecture: a recurrent Proximal Policy Optimisation (PPO) agent augmented with a supervised ''Scientist Head''. Operating within ''RuleSalad'', a novel and computationally lightweight Gym-MiniGrid benchmark featuring progressive levels of parametric and non-parametric procedural randomisation, the agent explicitly infers and classifies the hidden rules and reward objectives of the Partially Observable Markov Decision Process (POMDP) at each time step. By decoding the agent's internal belief state and measuring its Shannon entropy, we map mathematical uncertainty directly onto human-understandable metrics. Our framework visualises the exact moment the agent resolves epistemic uncertainty, empirically showing when it shifts from hypothesis-driven exploration to optimal exploitation. We empirically evaluate seven experimental conditions and find evidence of epistemic exploration at lower complexity levels, where within-episode belief entropy collapses to ≈0.55 nats within the first calibration steps. At higher combinatorial limits, we identify and diagnose two structural architectural boundaries: a grounding problem in semantic decoding, and an exploration-inference gap, arising from single-episode amnesiac resets under destructive probing.
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7214137018