Work

Percentage gain versus normalized gain for identifying training targets in medical record documentation education: a pre–post study

Year: 2026
Type: preprint
Abstract: Background Record audits are widely used to evaluate documentation training, and improvement is often compared across documentation domains using the percentage change in scores to judge which components worked and where to focus next. When post-training scores approach the instrument ceiling, this metric may mainly reflect baseline room for improvement. We compared percentage score gain with defect burden reduction, a normalized gain, and examined whether the defect profile changed after one training cycle. Methods We analysed blinded audit data from a self-paced micro-lecture program (53 lectures; 1327 min) for 49 physicians at a secondary-level hospital in China (December 2023 to July 2024). Five discharge records per physician were randomly sampled one month before and after training, de-identified, pooled, and scored on a 20-dimension instrument by four raters blinded to time point and author. Dimensions were ranked by each metric. Agreement between rankings, and between baseline and residual (post-training) defect burden, was assessed with Spearman correlation, with 95% uncertainty intervals (UIs) from 10 000 parametric simulations. Results The composite score increased from 89.14 ± 2.73 to 96.82 ± 1.38 ( P < 0.001), and post-training dimension scores converged within 96.04–97.97. Percentage score gain correlated with baseline score at r = − 0.960 (95% UI − 0.98 to − 0.90), versus r = − 0.649 (− 0.79 to − 0.35) for defect burden reduction. The two rankings were correlated ( r s = 0.707, 95% UI 0.52 to 0.90) but not interchangeable: only 2 of the top 5 dimensions overlapped, and 16 of 20 dimensions had rank intervals spanning more than 10 positions. Baseline and residual defect burden showed little agreement ( r s = 0.144, 95% UI − 0.33 to 0.43); informed consent moved from the lightest baseline burden to the second heaviest residual burden. Conclusions Near the instrument ceiling, percentage score gain largely reflects baseline room for improvement and should not be used alone to compare training effects across documentation domains. Normalized gain depends less on baseline, but dimension ranks are imprecise and are better reported with uncertainty intervals and in tiers. Targets for the next training cycle should be set from the residual defect burden re-measured after each cycle.
Language: English
Open Access status: green
OpenAlex ID: W7225880858
Cites: 25
Cited by:
Related to: 10