Year: 2026
Type: article
Abstract: Large language models (LLMs) such as ChatGPT have been widely adopted by chemistry undergraduate university students as a learning tool, but few methods exist to measure the scope of their influence on essay writing. This report introduces a protocol based on monitoring the frequency of specific words used excessively by LLMs among a sample of 1073 chemistry student essays between 2018 and 2025. Analysis revealed that 54 keywords known to be favored by LLMs simultaneously increased in frequency among essays submitted during the last two years. When the lists of keywords were applied to specific essays, the findings indicated 43% of essays submitted during 2025 had at least some evidence of LLM-generated text. This protocol offers a simple method to generate an approximate scope of LLM uptake among a cohort, which can be combined with student survey data to produce a detailed overall impression of students’ use of generative artificial intelligence. This work’s findings are intended to stimulate discussion on the best practices to update literature review assessment criteria, to ensure it is maintained as a valid assessment option within a curriculum equipped for the widespread adoption of LLMs.
Source: Journal of Chemical Education
Author: Keith Livingstone
Institution: University of Leeds
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
SDG: Quality education
Open Access status: hybrid
Estimated APC paid: $4,500
OpenAlex ID: W7218409369