Advice Is Not Effort: When a Language Model's Stated Preferences Predict Its Behaviour, Why the Answer Turns on the Channel a Preference Must Travel and the Kind of Preference Asked About, and the Positive Control a Null Result Owes
Year: 2026
Type: preprint
Abstract: (c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research Language models give coherent answers when asked which of two outcomes they prefer, and a growing literature asks whether those answers predict what the models then do. The results look contradictory. A 2026 study of five frontier models found that preferences over 72 entities of an undisclosed type, presented in the prompts as donation recipients, predicted donation advice almost perfectly (Spearman rho .94 to .98) and predicted refusal rates well, but predicted reading-comprehension accuracy weakly and in both directions, and agentic task success not at all. A second 2026 study found that offering models outcomes they rank highly does not improve the quality of their writing, while a plain exhortation to try harder does. In this analysis the author reviews and synthesises published results to show that the contradiction dissolves once two variables that the question leaves implicit are named. The first is the channel by which a preference would reach behaviour: selection among options the model is asked to choose between, withholding of help, or modulation of effort. The second is the kind of preference: an elicited taste over external objects, a commitment installed by post-training, or a stake in the model's own training or deployment. Elicited tastes reach selection and, more weakly, withholding; on effort, the published evidence is an accuracy difference of under one point on one benchmark, running in opposite directions in different models, and a controlled null. Trained commitments and self-regarding stakes do reach effort, in documented sandbagging and in quality that falls when success would serve a harmful cause. The paper also shows that the strongest correlations come from behaviour measured in nearly the same format as the preference, which the attitude-behaviour literature predicts without any appeal to motive. It consolidates the measured values in Table 1 and Figure 1, states an audit for preference-behaviour claims as Algorithm 1, and specifies seven tests, the first of which is a positive control that any null result on the effort channel owes.
Author: Mahendrakar, Pranay
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7233549428
Cites:
Cited by:
Related to:
