Year: 2026
Type: review
Study design: Systematic Review
Abstract: PURPOSE OF REVIEW: Large language models (LLMs) are increasingly being evaluated for clinical neurology applications, yet their real-world utility, safety, and deployment readiness remain unclear. This review maps the literature on LLMs applied to text-based and text-derived neurology tasks, characterizing current use cases, evaluation strategies, performance metrics, and reported safety concerns. We identify areas with relatively robust evidence, highlight important evidence gaps, and outline key methodological and clinical barriers that must be addressed before responsible clinical adoption. RECENT FINDINGS: Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR), we searched 4 databases (January 2022 to February 2026) and included 32 studies (2023-2026) spanning 7 neurology subspecialties; 9 were simulation or feasibility studies, 17 analyzed retrospective clinical data, and 6 reported prospective evaluation. Proprietary models predominated; 5 studies used retrieval-augmented generation and 1 used a multiagent framework. Performance was highest for constrained tasks, including binary diagnostic classification (area under the curve 0.75-0.94), information extraction (F1 0.89-0.90), patient-education question answering (accuracy 68%-97%), and ischemic stroke thrombectomy decision support (area under the curve [AUC] 0.92) and lowest for open-ended case-based reasoning (accuracy 42%-54%). Recurrent safety signals included hallucinated and fabricated citations, poor calibration with overconfidence (for example, 100% sensitivity with 27% specificity for epilepsy diagnosis), and a systematic bias toward intervention. On adapted Quality Assessment of Diagnostic Accuracy Studies Using Artificial Intelligence (QUADAS-AI) charting, overall risk of bias was low in 1 study (3%), moderate in 16 (50%), and high in 15 (47%), driven chiefly by absent external validation (31/32, 97%). Benchmarked against high-reliability and 6 Sigma standards, reported error rates remain orders of magnitude too high for autonomous use. SUMMARY: Evidence for LLMs in neurology is early, heterogeneous, and limited by scarce external and prospective validation. Current data support only narrow, clinician-supervised uses such as documentation and education, while cautioning against autonomous diagnostic or therapeutic use. Rigorous, prospective, externally validated studies with standardized reporting and explicit safety and ethical evaluation are needed before broader adoption. REGISTRATION: PROSPERO CRD420251082465.
Source: Neurology Clinical Practice
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7130360482
