Year: 2026
Type: conference-paper
Abstract: We investigate the degree to which human (and LLM) plausibility judgments of multiplechoice commonsense benchmark answers are subject to influence by (im)plausibility arguments for or against an answer, in particular, using rationales generated by LLMs.We collect 3, 000 plausibility judgments from humans and another 13, 600 judgments from LLMs.Overall, we observe increases and decreases in mean human plausibility ratings in the presence of LLM-generated PRO and CON rationales, respectively, suggesting that, on the whole, human judges find these rationales convincing.Experiments with LLMs reveal similar patterns of influence.Our findings demonstrate a novel use of LLMs for studying aspects of human cognition, while also raising practical concerns that, even in domains where humans are "experts" (i.e., common sense), LLMs have the potential to exert considerable influence on people's beliefs. 1
Institution: University of Maryland, College Park
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4416383073