A Competency Assessment Tool for Liver Contrast-Enhanced Ultrasound: Development and Validation in a Large Multicenter Retrospective Cohort of Trainees (Preprint)
Year: 2025
Type: article
Abstract: Background: Despite the increasing use of liver contrast-enhanced ultrasound (CEUS), no standardized, organ-specific tool exists for assessing trainees' cognitive competency. Existing instruments lack multidimensional evaluation and external validation, which limit training quality and patient safety. Objective: This study aims to develop and validate the CEUS Competency Assessment for the Liver (CCAL), a tool to standardize the evaluation of theoretical and cognitive skills in liver CEUS. Methods: This retrospective cohort study had four stages: (1) item generation from literature and guidelines, (2) 3 Delphi rounds (12/21, 57.1% female experts; mean age 51.0, SD 7.2 y) and cognitive testing (10 trainees) for content validity (>85% agreement) and clarity, (3) internal validation involving 100 sonography trainees (n=66, 66% assistant professors; n=27, 27% associate professors; n=7, 7% professors; n=70, 70% CEUS-naive) with 1 month or more of training at 4 hospitals (2018-2020), and (4) external validation on 579 sonography trainees (n=299, 51.6% assistant professors; n=183, 31.6% associate professors; n=97, 16.8% professors; n=439, 75.8% CEUS-naive) with 1 or more months of training at 25 multicenter sites (2018-2020). Primary outcomes were reliability (Cronbach α and Guttman split-half) and validity (exploratory factor analysis and item discrimination), while secondary outcomes included item difficulty distribution, pass rates, and between-group differences. Results: =.001). Competency gaps were identified across specific competency domains: the pass rate ranged from 32.5% (188/579; image-guided intervention) to 81.3% (471/579; indications and contraindications). Conclusions: The CCAL is the first liver-specific CEUS competency tool integrating objective knowledge testing with structured self-evaluation. Unlike generic or simulation-based tests, it is organ-specific and validated across a large multicenter cohort, providing a multidimensional framework that identifies specific competency gaps, most notably in advanced image-guided procedures, beyond global pass/fail outcomes. The tool supports competency-based medical education through rigorous data-driven assessment, targeted curricular improvements, cross-institutional benchmarking, and informed training standards. It evaluates cognitive competence exclusively and should not replace procedural assessment in high-stakes decisions. International validation and the establishment of criterion-related and predictive validity are essential next steps.
Source: JMIR Medical Education
DOI: 10.2196/86730
Authors: Yanping Ma, Bowen Zheng, Lei Shang, Hui Wang, Gaiqin Xue, Fang Nie, Yanli Guo, Yukun Luo, Hongxia Zhang, Xuan Wang, Xiang Jing, Ruifang Zhang, Jianjun Yuan, Rong Wu, Xiaomao Luo, Fenglin Dong, Yan Luo, Qiang Lu, Jintang Liao, Pintong Huang, Qin Chen, Jia’an Zhu, Youfeng Xu, Xiaoyan Xie, Jie Ren, Wen He
Institutions: Third Affiliated Hospital of Sun Yat-sen University, Air Force Medical University, Jilin University, Shanxi Provincial Cancer Hospital, Lanzhou University Second Hospital, Chinese PLA General Hospital, West China Hospital of Sichuan University, Capital Medical University, Beijing Tian Tan Hospital, Tianjin Third Central Hospital, First Affiliated Hospital of Zhengzhou University, Henan Provincial People's Hospital, Shanghai First People's Hospital, First Affiliated Hospital of Soochow University, Xiangya Hospital Central South University, Second Affiliated Hospital of Zhejiang University, Sichuan Academy of Medical Sciences & Sichuan Provincial People's Hospital, Peking University People's Hospital, Ningbo University Affiliated Hospital, Ningbo First Hospital, The First Affiliated Hospital, Sun Yat-sen University
Language: English
Subfield: Education
Field: Social Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $1,873
OpenAlex ID: W7166869060