Behavioral Probes for Identifying Proxy-Directed Behavior in Simple Gridworld: A Preliminary Empirical Study
Year: 2026
Type: preprint
Abstract: This preliminary empirical study investigates behavioral differences between intended-goal-reward and proxy-reward reinforcement learning agents in a simple gridworld environment. Two tabular Q-learning agents are trained in a 5 × 5 gridworld using different reward objectives and evaluated across five random seeds and five intended-goal locations. Evaluation measures include intended-goal success rate, cumulative return, proxy-region entry rate, and two behavioral indicators (PRI-A and PRI-Q). The results show contrasting reward-conditioned behaviors and demonstrate how the implemented behavioral indicators characterize action preferences in this controlled setting. However, the study does not establish goal misgeneralization following a distribution shift or demonstrate that the behavioral probes outperform conventional performance metrics. This work is presented as a preliminary empirical study.
Author: Jadhao, Maithili
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225995497
