Transportability of artificial intelligence models for predicting hip and knee osteoarthritis progression: a systematic review and meta-analysis of within-model changes in discrimination
Year: 2026
Type: preprint
Study designs: Systematic Review, Meta-Analysis
Abstract: Background Artificial intelligence (AI) models can achieve strong discrimination for osteoarthritis (OA) progression in development cohorts, but clinical translation depends on whether performance is maintained in new populations and settings. Previous OA reviews have mainly summarized absolute predictive performance rather than within-model changes after transport. Methods We searched PubMed, Embase, Web of Science Core Collection, and the Cochrane Library/CENTRAL from inception through 10 September 2026. We included reports evaluating AI or machine-learning models for future hip or knee OA outcomes and retained prespecified contextual evidence relevant to transportability. Validation was classified as independent external evaluation, internal-external or leave-one-cohort-out evaluation, quasi-external evaluation, or target-domain adaptation. The primary effect was the matched change in discrimination (ΔAUC/AUROC = AUC_external − AUC_internal). Risk of bias and applicability were assessed with PROBAST + AI. Random-effects synthesis used REML with publication-cluster robust inference; Hartung-Knapp, publication-level, leave-one-publication-out, and preprint sensitivity analyses were also performed. Results Of 14,171 records, 28 reports were included; 24 concerned knee OA and four hip OA. OAI was used in 24 reports and MOST in 15. The primary meta-analysis included 18 matched AUC/AUROC effects from five peer-reviewed publication clusters; 16 of 18 effects were negative. The pooled change was − 0.044 (publication-cluster robust 95% CI, − 0.064 to − 0.024; I² = 31.3%), with a similar effect-level estimate (− 0.044; 95% CI, − 0.062 to − 0.027). Adding one externally evaluated preprint yielded − 0.050 (95% CI, − 0.074 to − 0.026). All five publication-level mean changes were negative. Overall model-evaluation risk of bias was low in 7 reports, unclear in 10, and high in 11. External-domain evidence was heavily concentrated on OAI-to-MOST transport. Conclusions Across available matched comparisons, external evaluation was associated with an average reduction of approximately 0.044 AUC/AUROC units. Because the evidence base was small and repeatedly reused OAI and MOST, the pooled estimate should be interpreted as an empirical summary of available transport contrasts rather than a general transportability penalty. Future studies should evaluate frozen models in genuinely independent settings and report calibration and clinical utility alongside discrimination.
Source: Research Square
Institution: Guangzhou University of Chinese Medicine
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7225392647