Accuracy, information need, and run-to-run reliability of open-weight large language models for differential diagnosis in pet rabbits: a benchmark against confirmed cases.
Year: 2026
Type: other
Abstract: We evaluate two open-weight large language models (LLMs), run locally with fixed weights, and one commercial model as comparator (Claude Opus 5.5), on pet rabbits (Oryctolagus cuniculus) whose diagnosis was confirmed by histopathology, necropsy, surgery, pathogen detection, or unequivocal imaging. Cases are taken from veterinary dissertations and published case reports. Each case is presented at six cumulative levels of diagnostic information, in a structured and a narrative format with identical content, and each query is repeated five times. Each response is a ranked list of ten differential diagnoses, which veterinarians code against the confirmed diagnosis.
Source: Open Science Framework
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7223032209