Work

Who Punishes LLM Alignment? User Characteristics and the Evaluation of Ethical Response Refusals

Year: 2026
Type: article
Abstract: Ethical refusals are essential for aligning large language models (LLMs) with safety constraints, yet users often penalize them in comparative evaluations. Whether this refusal penalty varies systematically across users remains unclear. Drawing on human–computer interaction research, we examine how users’ evaluative priorities and usage intensity shape responses to ethical refusals. Using the PRISM alignment dataset, which links LLM evaluations to user characteristics, we find that safety preferences significantly mitigate refusal penalties, whereas preferences for helpfulness, fluency, and creativity do not. Heavy users penalize refusals more strongly, an association substantially mediated by lower safety preferences. These findings suggest that acceptance of safety-aligned behavior depends primarily on users’ safety orientation rather than preferences for instrumental response qualities. Our results highlight the limitations of evaluating LLM safety alignment through aggregate user preferences and underscore the importance of accounting for user heterogeneity in evaluation benchmarks and deployment decisions.
Source: Figshare
Language: English
Open Access status: green
OpenAlex ID: W7236266394
FWCI:
Cites:
Cited by:
Related to: 10