What AI editing changes in scientific abstracts, and what remains when the AI is asked to sound human
Year: 2026
Type: article
Abstract: Many researchers now use artificial intelligence (AI) tools such as ChatGPT or Claude to edit their papers. We asked what such editing changes in the wording of scientific abstracts, and whether the changes disappear when the AI is told to sound human. In this controlled experiment, we used 1,501 abstracts published in six dental journals in 2021, before AI writing tools were widely available. An AI model (Claude, version claude-sonnet-4-6) rewrote each abstract twice: once with a standard editing instruction, and once with an added instruction to avoid wording that sounds machine-made. This gave three versions of every abstract. Rewriting was done in June 2026 and the analysis in July 2026. Using 60% of the abstracts, we listed two- to four-word sequences that the AI used much more often than the original authors, and counted them in the remaining 40%. We also measured word and sentence length, sentence complexity and readability. Some word sequences became far more common after AI editing. For example, “these findings” appeared 28 times in the original abstracts used to make the list and 287 times in their AI-edited versions. In the abstracts set aside for testing, the AI-edited versions contained about 29 of the listed two-word sequences per 1,000 words, compared with about 10 in the originals (Cohen’s d 1.40; values above 0.8 are considered large). AI editing also made the abstracts harder to read, with longer words and sentences. Word choice and sentence structure each carried partly different information. When the two were combined, a randomly chosen AI-edited abstract scored higher than a randomly chosen original in about 96 of 100 comparisons (area under the curve, 0.96). Telling the AI to sound human reduced the differences in word choice by only 10% to 32%. AI editing left recognizable traces in the wording and structure of scientific abstracts, and asking the AI to sound human removed only part of them. These traces describe groups of texts and cannot show whether a particular paper was written with AI. Open disclosure of AI use remains more reliable than attempts to detect it.
Language: English
Subfield: Library and Information Sciences
Field: Social Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $2,490
OpenAlex ID: W7235146580
