Who Punishes LLM Alignment? User Characteristics and the Evaluation of Ethical Response Refusals
Year: 2026
Type: article
Abstract: Ethical refusals are essential for aligning large language models (LLMs) with safety constraints, yet users often penalize them in comparative evaluations. Whether this refusal penalty varies systematically across users remains unclear. Drawing on human–computer interaction research, we examine how users’ evaluative priorities and usage intensity shape responses to ethical refusals. Using the PRISM alignment dataset, which links LLM evaluations to user characteristics, we find that safety preferences significantly mitigate refusal penalties, whereas preferences for helpfulness, fluency, and creativity do not. Heavy users penalize refusals more strongly, an association substantially mediated by lower safety preferences. These findings suggest that acceptance of safety-aligned behavior depends primarily on users’ safety orientation rather than preferences for instrumental response qualities. Our results highlight the limitations of evaluating LLM safety alignment through aggregate user preferences and underscore the importance of accounting for user heterogeneity in evaluation benchmarks and deployment decisions.
Source: Figshare
Authors: Stefan Pasch, Min Chul Cha
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7236266394
