Year: 2026
Type: article
Abstract: As large language models (LLMs) increasingly generate human-like text, it is becoming an increasingly pressing empirical and societal question: how can AI-generated text be distinguished from human-written text? Through two studies, this paper explored whether and how people judge AI authorship. In Study 1, participants (n = 101) judged the written responses to an inside joke prompt (the prompt chosen based on LLMs' response) and assessed each response on a 7-point scale of −3, definitely AI-written, to +3, definitely human-written, followed by self-reports of justifications. People's classifications were near chance accuracy (51.1%), but their continuous ratings did separate the human-written from the AI-written responses. Multivariate logistic regression showed that accuracy was best predicted by attention to surface-level errors and structural features rather than by deeper experiential cues (for example, interpretations of humor and personal schema matches). In Study 2, a new sample (n = 100) evaluated AI responses deliberately modified with spelling and grammatical errors. Accuracy dropped steeply in participants below chance (39.2%), validating the conclusion of Study 1. Together, the results of these two studies reveal that people's capacity to detect human expression rests less on detecting genuine lived experience than on superficial linguistic cues that are easy to manipulate.
Author: Yijia Xu
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7235278543
