Year: 2026
Type: conference-paper
Abstract: Large Language Models (LLMs) are highly effective at adapting to users' styles, preferences, and contextual signals-a property that underlies much of their practical usefulness, but which can even manifest as sycophancy, i.e., alignment with user-implied beliefs even when these contradict factual accuracy or rational reasoning.Prior work treats sycophancy as a surface-level artefact addressed via inferencetime or post-hoc methods.We argue that it is a policy-level failure arising from a lack of agentic control over agreement under pressure.To make sycophancy tractable to explicit control, we propose learning agentic policies modelling LLMs' behaviour as a decision-making problem.We equip the model with an explicit action space that includes countering misleading signals, asking for clarification or, when appropriate, adapting to the user.The policy is trained via control coefficients to optimise a multi-objective reward balancing task success and sycophancy resistance.Crucially, at inference, a meta-policy infers these parameters from the context.We evaluate the proposed method across different benchmarks, demonstrating that it reduces sycophancy, improves performance, and generalises robustly across languages.These findings suggest that mitigating sycophancy requires moving beyond compliance-oriented generation towards agreement-agentic control.
Authors: Leonardo Ranaldi, Giulia Pucci
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: bronze
OpenAlex ID: W7140142374