Benchmarking the performance of large language models on medical genetics questions: a comparative evaluation of GPT-5.1, Gemini Pro 3, and Claude Opus 4.5
Year: 2026
Type: article
Abstract: Objective This study benchmarks the performance of three state of the art large reasoning models comprising GPT 5.1, Gemini Pro 3, and Claude Opus 4.5 by evaluating their accuracy, response consistency, processing speed, and output length on medical genetics questions from the Turkish Medical Specialty Examination (TUS). Methods A total of 233 medical genetics questions were selected based on predefined inclusion criteria covering molecular mechanisms, inheritance patterns, genetic disorders, and diagnostic techniques. Each question was independently submitted to GPT 5.1, Gemini Pro 3, and Claude Opus 4.5 using a standardized prompt. Model accuracy was compared using McNemar's test. Response consistency was assessed using Cohen's Kappa. Response time and word count were analyzed using the Friedman test with Dunn Bonferroni post hoc comparisons. Results No statistically significant differences were observed in overall accuracy among the models ( p > 0.05). GPT 5.1 achieved the highest accuracy at 97.9 percent followed by Claude Opus 4.5 at 96.6 percent and Gemini Pro 3 at 96.1 percent. Cohen's Kappa indicated moderate qualitative divergence between Gemini Pro 3 and Claude Opus 4.5 with kappa equal to 0.451. Significant differences were found in response time and output length ( p < 0.001). Conclusion Although all models demonstrate similarly high accuracy, they differ in operational behavior, suggesting distinct internal knowledge representations and response strategies.
Source: Frontiers in Medicine
Authors: Muserref Basdemirci, Melek Tassoker
Institutions: Konya City Hospital, Necmettin Erbakan University
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $3,801
OpenAlex ID: W7221703787