External validation and adaptation of the Computational Barthel Index Tool in a large nursing home cohort
Year: 2026
Type: article
Abstract: The Computational Barthel Index Tool (CBIT), originally developed using U.S. Department of Veterans Affairs (VA) data, predicts future dependency in activities of daily living up to one year in advance. This study externally validated CBIT in a large non-VA nursing home population and evaluated whether population-specific adaptation could recover performance following external transport. We analyzed SEER-Medicare linked data, including 843,836 patients and over 8 million Minimum Data Set (MDS) assessments. The predictor and outcome structures of the original CBIT models were reconstructed in the SEER-Medicare cohort. Pretrained models were evaluated without modification for discrimination and calibration at current, 3-, 6-, and 12-month prediction timeframes. Population-specific adaptation was evaluated using progressively larger subsets of SEER-Medicare training data and compared with model reconstruction in the SEER-Medicare. Under strict external validation, discrimination declined relative to the original VA models, particularly for longer-term predictions. For evaluation models, mean AUC decreased from 0.795 in the original models to 0.693 at current prediction, 0.595 at 3 months, 0.595 at 6 months, and 0.592 at 12 months. Re-evaluation models showed stronger but similarly declining external performance. Calibration error increased, particularly for evaluation models at longer timeframes. Subgroup analyses showed modest variation in discrimination across demographic and clinical groups. Population-specific adaptation substantially recovered discrimination, with performance improving as more SEER-Medicare training data were incorporated. Using the adaptation training set, adapted models achieved performance comparable to SEER-Medicare model reconstruction. External validation revealed substantial performance degradation when CBIT was transported to a non-VA population. Population-specific adaptation recovered much of this performance loss, supporting a structured approach to clinical ML transportability in which pretrained models are externally validated first, followed by adaptation when performance degradation is observed. Full model reconstruction remains an alternative when sufficient local data are available.
Source: PLOS Digital Health
Institution: George Mason University
Language: English
Subfield: Geriatrics and Gerontology
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $3,043
Awards: G1352184354, G1395202410, G2337224928, CALIFORNIA COORDINATED CANCER PREVENTION AND CONTROL PROGRAM; CALFORNIA COMPREHENSIVE CANCER CONTROL PROGRAM; CALIFORNIA CANCER REGISTRY - THE CALIFORNIA DEPARTMENT OF PUBLIC HEALTH (CDPH) HAS CONDUCTED CANCER PREVENTION AND CONTROL ACTIVITIES FUNDED BY THE CENTERS FOR DISEASE CONTROL FOR MORE THAN 35 YEARS. THESE INCLUDE HIGH PERFORMING, NATIONALLY RECOGNIZED PROGRAMS, INCLUDING THE CALIFORNIA CANCER REGISTRY (CCR) AND THE CALIFORNIA COMPREHENSIVE CANCER CONTROL PROGRAM (CCCCP). THESE PROGRAMS SAVE LIVES OF CALIFORNIANS BY PREVENTING AND REDUCING THE DEVASTATING EFFECTS OF CANCER ACROSS THE CANCER CONTROL CONTINUUM (PREVENTION TO SURVIVORSHIP) THROUGH ADVANCING POLICY, EDUCATION, EARLY DETECTION, DIAGNOSIS, TREATMENT, AND SURVIVORSHIP WITH AN INTEGRATED FOCUS ON REDUCING HEALTH DISPARITIES. AT ITS CORE, THE PURPOSE OF CCR IS TO MONITOR THE BURDEN OF CANCER ACROSS THE ENTIRE STATE, ASSIST IN CANCER CONTROL PROGRAM EVALUATION AND GUIDANCE, RESPOND TO PUBLIC CONCERNS ABOUT CANCER, AND PROVIDE HIGH QUALITY STRUCTURED DATA IN SUPPORT OF RESEARCH, PREVENTIVE AND CLINICAL CARE, AND IMPROVING CANCER OUTCOMES FOR ALL SURVIVORS. CCCCP’S OVERARCHING GOALS ARE TO PROVIDE THE INFRASTRUCTURE TO ENSURE COORDINATION OF CANCER CONTROL EFFORTS IN CALIFORNIA, ENGAGEMENT OF ALL CANCER CONTROL CONSTITUENTS THROUGH THE CALIFORNIA DIALOGUE ON CANCER AND DEVELOP OUR STATE’S CANCER CONTROL PLAN TO GUIDE AND EVALUATE OTHER PROGRAMS’ AND STAKEHOLDERS’ ACTIVITIES BASED ON THE LATEST EVIDENCE BASED STRATEGIES. THE TWO PROGRAMS DESCRIBED IN THIS PROPOSAL ARE HIGHLY COMMITTED TO COLLABORATION AMONG THEMSELVES, WITH THE CALIFORNIA EARLY BEAST AND CERVICAL CANCER SCREENING PROGRAM, AS WELL AS OTHER PARTNERS BOTH INSIDE AND OUTSIDE THE STATE. BASED ON THESE PRINCIPLES, THE CCR AND CCCP WILL SUCCESSFULLY APPLY THE CENTERS FOR DISEASE CONTROL AND PREVENTION PROGRAM STANDARDS TO ACHIEVE THE ULTIMATE GOAL OF REDUCING THE CANCER BURDEN OF CALIFORNIA AND THE NATION., G2612351997, G3448447748, G3509688676, G4332884407, G4428815567, G4854421162, G504639888, G6951404960, G7290139498, G8083727551, G8509442309, G946791773
OpenAlex ID: W7223281619