Work

Return to sport decision support with large language models after lower limb ligament injury

Year: 2026
Type: article
Abstract: Patients and clinicians increasingly use large language models (LLMs) for return-to-sport decisions, but their reliability and the effect of structured clinical context remain uncertain. In this pre-registered, blinded, multi-model study, GPT-5, Gemini 2.5 Pro, Claude Sonnet 4.5, and DeepSeek-V3.2-Exp answered 12 fictional anterior cruciate ligament reconstruction or lateral ankle sprain vignettes under instruction-only (Prompt+) and guideline-derived context (Context+) conditions across two runs, generating 192 outputs. Three blinded domain experts scored six binary criteria. The primary outcome was a correct clinical claim supported by a valid DOI or PMID. Context + did not improve the primary outcome compared with Prompt + (OR 0.41, 95% CI 0.16–1.05; p = 0.065). GPT-5 was the only model whose point estimate favoured Context+, whereas the other three models favoured Prompt+. Claude Sonnet 4.5 showed a higher majority-vote Hallucination rate under Context + (22.7% vs 62.5%; Holm-adjusted p = 0.014); however, inter-rater agreement for Hallucination was poor (Fleiss’ κ < 0 in both domains), so this finding should be interpreted cautiously. Citation validity declined under Context + in three of the four models. Structured guideline-derived context did not reliably improve return-to-sport decision support in the models and conditions tested, and its effects varied between models. LLMs intended for clinical decision support should therefore undergo task- and model-specific evaluation before deployment.
Language: English
Field: Medicine
Open Access status: hybrid
Estimated APC paid: $3,650
OpenAlex ID: W7234053608
FWCI:
Cites: 32
Cited by:
Related to: 10