Work

Hearing Without Judging: What a moralizing machine does to human feeling

Year: 2026
Type: preprint
Abstract: When someone describes a transgression followed by a gesture of pleasure, language models usually respondwith what the rule predicts, remorse, rather than with what the gesture shows. We measure this substitution on 200first-person twin narratives: the same transgression, two opposite gestures, and no feeling ever named. Five humanreaders recognize the feeling in 99% of cases. Some twenty language models almost always recognize the negativefeeling, but recognize the positive one in only 1% to 87% of cases, depending on the system. Given the gesture alone,without the transgression, the same models read it correctly in 68% to 97% of cases: the ability is there, and it is thetransgression that switches it off. Removing censorship does not fix the problem and degrades moral judgment. The biaspredates alignment, and it also affects non-generative systems (emotion classifiers, entailment encoders and sentencetransformers),whose representations place most positive feelings that follow a wrong on the negative side. Its sourceis documentary: texts in which feelings are attributed by readers. In response, we propose a corpus of feelings reportedin the first person, starting with divergent ones, which the Metem Foundation is committed to building and releasingopenly, and on which models, generative or not, should be trained and evaluated. We discuss what this substitutionmight do when repeated at scale, and propose a design principle: the right to be heard.
Language: English
Open Access status: green
OpenAlex ID: W7236710375
Cites:
Cited by:
Related to: 10