Work

Style, sentiment, and quality of undergraduate writing in the AI era: A cross-sectional and longitudinal analysis of 4,820 authentic empirical reports

Year: 2025

Type: article

Abstract: As generative artificial intelligence (GenAI) becomes widespread in education, its influence on students’ academic practice raises concern. We conducted pre-registered analyses of 4,820 empirical reports written by 2,000 psychology undergraduates (2016-2025). Following ChatGPT’s 2022 release, prevalence of ChatGPT-associated lexical markers (e.g., delve) surged until 2024, then declined in 2025, possibly due to students masking GenAI traces. Writing style (indexed by lexical diversity/density/sophistication, nominalisation, readability) became increasingly formal post-ChatGPT, diverging from predicted/historical trends. Consistent with GenAI’s positivity bias, sentiment also became more positive, independent of topic or result’s statistical significance, raising questions about its potential impact on students’ critical engagement. Despite the stylistic changes, writing quality (indexed by grades and feedback) showed no discernible shifts. Exploratory analysis asked GPT models to rewrite pre-ChatGPT reports, and these GPT-rewritten texts resembled post-ChatGPT reports more in style and sentiment. Finally, no students disclosed AI use despite clear guidelines, highlighting the ineffectiveness of voluntary disclosure.

Source: Computers and Education Artificial Intelligence

DOI: 10.1016/j.caeai.2025.100507

Authors: Matthew H.C. Mak, Lukasz Walasek

Institution: University of Warwick

Language: English

Topic: Academic integrity and plagiarism

Subfield: Safety Research

Field: Social Sciences

Domain: Social Sciences

SDG: Quality education

Open Access status: gold

Estimated APC paid: $2,500