Three versions of an atopic dermatitis case report written by humans, artificial intelligence, or both: Identification of authorship and preferences
Year: 2024
Type: article
Study design: Observational Study
Abstract: Background The use of artificial intelligence (AI) in scientific writing is rapidly increasing, raising concerns about authorship identification, content quality, and writing efficiency. Objectives This study investigates the real-world impact of ChatGPT, a large language model, on those aspects in a simulated publication scenario. Methods Forty-eight individuals representing 3 medical expertise levels (medical students, residents, and experts in allergy or dermatology) evaluated 3 blinded versions of an atopic dermatitis case report: one each human written (HUM), AI generated (AI), and combined written (COM). The survey assessed authorship, ranked their preference, and graded 13 quality criteria for each text. Time taken to generate each manuscript was also recorded. Results Authorship identification accuracy mirrored the odds at 33%. Expert participants (50.9%) demonstrated significantly higher accuracy compared to residents (27.7%) and students (19.6%, P < .001). Participants favored AI-assisted versions (AI and COM) over HUM ( P < .001), with COM receiving the highest quality scores. COM and AI achieved 83.8% and 84.3% reduction in writing time, respectively, compared to HUM, while showing 13.9% ( P < .001) and 11.1% improvement in quality ( P < .001), respectively. However, experts assigned the lowest score for the references of the AI manuscript, potentially hindering its publication. Conclusion AI can deceptively mimic human writing, particularly for less experienced readers. Although AI-assisted writing is appealing and offers significant time savings, human oversight remains crucial to ensure accuracy, ethical considerations, and optimal quality. These findings underscore the need for transparency in AI use and highlight the potential of human-AI collaboration in the future of scientific writing.
Authors: Mara Giavina Bianchi, Andrew D’adario, Pedro Giavina Bianchi, Birajara Soares Machado, Rosana Agondi, Stephanie K.A. Almeida, Wandilson Xavier Alves Junior, Larissa M. Armelin, Marcelo Vivolo Aun, Natália Bordignon, Karla Boufleur, Felipe B. Brunheroto, Elisabeth A. Callegaro, Paula Lazaretti M. Castro, Herberto Jose Chong-Neto, Mariana D. Dall’Osto, Julia Abou Dias, Viviane Heintze Ferreira, André Luiz Oliveira Feodrippe, Livia G. Fonseca, Clycia M. Garcia, Bruna H. Giavina-Bianchi, Ekaterini Goudouris, Danilo Gois Gonçalves, Debora D. Hernandes, Malek Imad, Larissa S. Izabel, Lucas Cauê Jacintho, Carolina Khouri-Panzarin, Fabio Kuschnir, Maria Beatriz Pádua Lima, Amanda I. Lopes, Larissa Nathalia Macêdo Nóbrega Lopes, Alice Rocha de Magalhães, Eli Mansour, Ana Karolina B.B. Marinho, Vivian S. Martimiano, Pedro H. Milori, Antonio Marcondes Mutarelli, Guilherme Paes Gonçalves Nogueira, Beatriz K.T. Oguido, Bruna S. Alarcon de Oliveira, Emerson Costa de Oliveira, Georgia A. Padulla, Letícia D’Ordaz Lhano Santos, Micaelly Samara Meneses Santos, Emanuel Sarinho, Marcela Schoen, Brian Lucas A. Sousa, Eduardo Magalhães de Souza-Lima, Beatriz C. Todt, Najla Braz da Silva Vaz
Institution: Universidade de São Paulo
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $2,000
OpenAlex ID: W4404733678