Assessing the quality of AI-assisted peer reviews: A comparative assessment of ChatGPT, Claude Sonnet and Google Gemini in reviewing scientific articles
Year: 2026
Type: article
Abstract: Objectives: Peer review validates research findings and upholds the quality and credibility of scientific publications. This study aimed to assess the quality of peer-review reports generated by GPT-5.6, Claude Sonnet-4.6, and Google Gemini-3.1 Pro, while evaluating the same set of published scientific articles. Methodology: This comparative study was performed in the “Department of Physiology, College of Medicine, King Saud University, Riyadh, Saudi Arabia” from August 23, 2026, to September 10, 2026. In this study, 30 original published articles were identified, and each article was submitted to GPT-5.6 (n=30), Claude Sonnet-4.6 (n=30), and Google Gemini-3.1 Pro (n=30). Each AI model generated 30 individual review reports, for a total of 90 peer-review reports. A panel of five faculty members evaluated the peer-review reports, and each report was scored out of five marks. Each examiner evaluated peer-review reports generated by ChatGPT (30), Claude Sonnet (30), and Google Gemini (30). The frequencies, percentages, mean ± SEM and 95% CIs were calculated, and p-values < 0.05 were considered statistically significant.Results: AI models generated 90 peer-review reports, 30 each from ChatGPT, Claude Sonnet, and Google Gemini. Across the five examiners, ChatGPT received an overall grade of 634.1/750 (84.54%), Claude Sonnet received 690.4/750 (92.05%), and Google Gemini received 465.3/750 (62.04%). Claude’s Sonnet review reports received the highest grades, followed by ChatGPT and Google Gemini (p=0.001). Conclusions: The AI model, Claude Sonnet-4.6, produces significantly better review reports than GPT-5.6 and Google Gemini-3.1 Pro. These AI models summarize complex information and identify potential gaps, methodological concerns, and issues in the results and discussion sections of medical science articles. Integrating AI models within professional boundaries supports academic journals’ editorial workflows rather than replacing human peer review.
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $400
Funder: King Saud University
OpenAlex ID: W7236672406
