Work

Agent-G2: Gaussian Guidance for Agentic Reinforcement Learning

Year: 2026

Type: conference-paper

Abstract: Hint-based reinforcement learning addresses reward sparsity in long-horizon agentic tasks by retaining a prefix of an expert trajectory before each rollout, letting the policy explore from a state closer to success.Its effectiveness hinges on the guidance depth: how much of the trajectory to keep.Existing methods treat this depth as a deterministic scalar.Scheduled approaches share one value across samples and ignore per-task heterogeneity; persample probing estimates it separately at the cost of extra rollouts.We find that useful guidance occupies a band of depths whose informativeness profile is approximately Gaussian around the band center, rather than concentrating at a single optimal point.We propose Agent-G 2 , a Gaussian guidance framework that draws the depth per task from a Gaussian whose center and spread are estimated online from rollouts already collected for policy optimization, requiring no probe rollouts or learned depth predictor.The center combines a global baseline with per-cluster difficulty, and the spread tracks within-cluster variance.We evaluate Agent-G 2 on ALFWorld and WebShop on Qwen2.5-1.5B/ 7B-Instruct.Agent-G 2 outperforms the strongest hint-based, hint-free, and Aux-RL baselines on ALFWorld by 2.3 / 3.9 / 7.4 points at under one-third the rollout cost of per-sample probing.Code is available at https://github.com/ZJU-REAL/Agent-G2.

Source: Conference on Empirical Methods in Natural Language Processing (EMNLP)

DOI: 10.18653/2026.emnlp-main.268