Evaluation of the Performance of Chatgpt/artificial Intelligence in the Multiple-choice Test to Obtain the Title of Specialist in Orthopedics and Traumatology
Year: 2025
Type: article
Abstract: Introduction: ChatGPT, an advanced Artificial Intelligence model specialized in natural language processing, shows remarkable abilities, achieving high scores in certification exams in various specialties. This study aims to evaluate ChatGPT's performance in multiple-choice tests applied to obtain specialist certification in Orthopedics and Traumatology. Methods: 2022 (TEOT) (Specialist in Orthopedics and Traumatology Test). We excluded non-text-based questions. Each question was entered individually into ChatGPT, with a new session initiated for each question. Performance was evaluated regarding number of words and questions' taxonomic classification. Results: Of the 95 questions analyzed, ChatGPT answered 61.05% correctly and 38.95% incorrectly. There was no statistically significant difference regarding number of words, and ChatGPT's performance did not vary according to taxonomic level. Conclusion: ChatGPT demonstrated vast knowledge in Orthopedics, with acceptable performance in the TEOT exam. Results suggest ChatGPT's an educational and clinical resource in Orthopedics, but needs future progress and human supervision for its effective application.
Source: Acta Ortopédica Brasileira
Authors: LUCAS PLENS DE BRITTO COSTA, DANILO HENRIQUE PIZZO DE CASTRO, RENATO PINHEIRO CORDEIRO, RÔMULO BALLARIN ALBINO
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $600
OpenAlex ID: W4409217199