Large language models in rheumatology: applications, methodological quality and clinical translation – a systematic review
Year: 2026
Type: review
Study design: Systematic Review
Abstract: Objectives To map applications of large language models (LLMs) in rheumatology and evaluate the methodological quality and clinical translation of the evidence.Methods MEDLINE, Embase and Web of Science were searched from inception to 13 July 2026 for primary studies of LLMs in rheumatology. Two reviewers independently screened records and analysed data. Each study was assigned to one application domain and to the highest level of a review-derived translational maturity framework. Risk of bias was assessed with the Alberta Quality Assessment Tool: Risk of Bias (AQAT:RoB; worst-item rule), where human evaluation of free-text answers was reported; findings were synthesised narratively.Results 93 studies were included, 79 (84.9%) from 2025 or 2026. Patient education (39, 41.9%) and clinical decision support (38, 40.9%) dominated. Accuracy in clinical decision support was strongly task-dependent, with medians of 90.9% for structured diagnostic tasks and 40.0% for open-ended differential diagnosis (best-performing model per study). Patient-facing output had a median Flesch-Kincaid grade of 12.2; none of the 13 readability studies met the recommended sixth-grade target. 22 studies (23.7%) used real-world clinical data, mostly retrospectively; only four (4.3%) evaluated an LLM prospectively in a clinical workflow. Prompts were fully reported in 59.1% of studies, and answer stability was tested in 19.4%. Among the 68 studies amenable to AQAT:RoB assessment, five (7.4%) were at low risk of bias.Conclusions LLM research in rheumatology is expanding rapidly but remains dominated by prompt-response benchmarks with surrogate outcomes, incomplete reporting and, among AQAT:RoB-assessable studies, predominantly unclear or high risk of bias. Whether LLMs improve rheumatology care remains unresolved; prospective evaluations with patient-relevant endpoints are required.
Source: RMD Open
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $2,391
OpenAlex ID: W7224098905