Work

Behavioral Probes for Identifying Proxy-Directed Behavior in Simple Gridworld: A Preliminary Empirical Study

Year: 2026
Type: preprint
Abstract: This preliminary empirical study investigates behavioral differences between intended-goal-reward and proxy-reward reinforcement learning agents in a simple gridworld environment. Two tabular Q-learning agents are trained in a 5 × 5 gridworld using different reward objectives and evaluated across five random seeds and five intended-goal locations. Evaluation measures include intended-goal success rate, cumulative return, proxy-region entry rate, and two behavioral indicators (PRI-A and PRI-Q). The results show contrasting reward-conditioned behaviors and demonstrate how the implemented behavioral indicators characterize action preferences in this controlled setting. However, the study does not establish goal misgeneralization following a distribution shift or demonstrate that the behavioral probes outperform conventional performance metrics. This work is presented as a preliminary empirical study.
Language: English
Open Access status: green
OpenAlex ID: W7225995497
Cites:
Cited by:
Related to: 10