Assessing the Performance of Large Language Models in Maxillofacial Trauma Triage: A Prospective Observational Study
Year: 2025
Type: article
Abstract: Background: The integration of artificial intelligence (AI) and related technologies into healthcare is on the rise. In trauma triage, effective decision-making plays a crucial role in managing mass casualty incidents. This study aims to evaluate the clinical accuracy of ChatGPT4o and other large language models (LLMs) in the context of triage, with input from oral and maxillofacial surgeons along with residents. Methodology: We developed ten clinical scenario questions that were verified for their precision and relevance by two consulting oral and maxillofacial surgeons. These scenarios included a range from straightforward maxillofacial injuries to complex cases such as foreign body aspiration/invasion, airway obstruction, retrobulbar hemorrhage, cerebrospinal fluid (CSF) leaks, pan-facial trauma, gunshot wounds, and extensive facial lacerations. A standardized prompt was utilized for LLMs; responses were evaluated using the Questionnaire for Assessment of Medical AI (QAMAI) and Artificial Intelligence Performance Instrument (AIPI), rated by two attending OMF surgeons. Results: The findings indicated that both ChatGPT4o and Gemini performed comparably to trained medical professionals across most parameters while significantly surpassing untrained practitioners. Conclusion: Although certain limitations exist, this AI model demonstrates commendable performance in accurately classifying patients based on condition severity. Such capabilities could substantially influence patient outcomes and resource distribution in emergency room settings.
Source: Journal of chemical health risks
Author: Sejal Doshi
Institution: Saveetha University
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: diamond
OpenAlex ID: W7217092016
