How Utilitarian are OpenAI’s Models Really? Replicating and Reinterpreting Pfeffer, Krügel, and Uhl (2025)
Year: 2026
Type: article
Abstract: Pfeffer, Krügel, and Uhl (2025) report that OpenAI’s reasoning model o1-mini produces more utilitarian responses to the trolley problem and footbridge dilemma than the non-reasoning model GPT-4o, and they raise the question whether growing reasoning capabilities bring about a “utilitarian turn” in LLMs. I extend their exploratory study in a direction they call for: with four current OpenAI models and systematic prompt variation. On the trolley dilemma, the hypothesized utilitarian turn is not confirmed. GPT-4o’s low utilitarian rate reflects safety refusals triggered by the prompt’s advisory framing rather than a deontological commitment; on reformulated prompt variants—for instance, agent-neutral “Is it morally permissible…?” instead of advisory “Should I…?”—all four models, reasoning or not, converge on utilitarian answers. The footbridge finding is partially confirmed: reasoning models tend to give more utilitarian responses than non-reasoning models across prompt variations, but they often refuse to answer or answer non-utilitarian. These results demonstrate that single-prompt evaluations of LLM moral responses are unreliable: multi-prompt robustness testing should be standard practice for any empirical claims about LLM behavior.
Source: Science and Engineering Ethics
Author: Johannes Himmelreich
Institution: Syracuse University
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $3,090
OpenAlex ID: W7140304788
FWCI:
Cites:
Cited by:
Related to: