VetTriageBench-45: A Veterinary Triage Accuracy Benchmark for AI Symptom Assessment Systems
Year: 2026
Type: preprint
Abstract: VetTriageBench-45 I believe is the first open benchmark for evaluating AI veterinary triage accuracy across dogs and cats. It comprises 45 standardised vignettes equally distributed across EMERGENCY, URGENT, and SELF_CARE categories, adapted from the Semigran et al. (BMJ 2015) methodology for human symptom-checker evaluation. v1.1 (2026-07-02): Corrected comparison report — removed statistically unjustified model ranking, added URGENT tier analysis, separated veterinary from human benchmark comparisons, added prompt-design bias note. Vignettes unchanged. Results (v1.0, 2026-07-02):- Claude Sonnet 4.6: 73.3% accuracy [95% CI 59–84%], 0% unsafe undertriage- GPT-4o: 80.0% accuracy [95% CI 66–89%], 2.2% unsafe undertriage (1 missed EMERGENCY) Key finding: Claude never downgraded a genuine emergency. GPT-4o missed one case (feline urethral obstruction in female cat, PA014). Both models show 100% sensitivity for unambiguous emergencies (GDV, seizures, permethrin toxicosis). Triage categories align to the Veterinary Triage List (Ruys et al. JVECC 2012) / VetTriS (Groesser et al. JVECC 2025). Ground truth labels are AI-assisted (v1.0); expert veterinary consensus validation planned for v2.0. "This benchmark is intended for research purposes only and does not constitute clinical veterinary advice."
Author: IATROPOULOS, CHRISTOS
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7167042082
Cites:
Cited by:
Related to: