Comparative Analysis of Large Language Models in First-Aid Scenario Recognition and Management: An In Silico Evaluation of ChatGPT and Claude
Year: 2025
Type: article
Abstract: INTRODUCTION: Large language models (LLMs) deliver real-time, conversational guidance, yet their reliability for time-critical first aid remains unclear. MATERIALS AND METHODS: Five standardized vignettes (drowning, animal bite, opioid overdose, lightning strike, and frostbite) were presented three times each to GPT-4o (OpenAI, San Francisco, CA, USA) and Claude 3.5 Sonnet (Anthropic, San Francisco, CA, USA). Outputs were scored (0 = incorrect/unsafe, 1 = incomplete, 2 = entirely correct) across six domains: diagnostic accuracy, first-aid advice, triage accuracy, comprehensiveness, safety, and consistency. Scores were averaged within and across vignettes. RESULTS: Both LLMs achieved perfect diagnostic (2.0) and triage (2.0) scores. Claude 3.5 outperformed GPT-4o in first-aid accuracy (2.0 vs 1.5), comprehensiveness (1.5 vs 1.3), and consistency (2.0 vs 1.6). Safety ratings were comparable (1.9-2.0). Key GPT-4 omissions included naloxone administration for opioid overdose and immediate sheltering guidance after a lightning strike. CONCLUSIONS: Claude 3.5 provided more complete and stable first-aid guidance than GPT-4, although both models reliably identified emergencies and advised on the appropriate escalation of care. Wider implementation warrants larger vignette sets, real-user simulations, and continuous monitoring for guideline concordance.
Source: Cureus
DOI: 10.7759/cureus.94229
Authors: Norvin T West, Ajani J Edwards, Jessica K Sims, Jordan E O'Brien, Jeffrey S Upperman
Institution: Vanderbilt University Medical Center
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: diamond
OpenAlex ID: W4415002833