When the grade disappears: A natural experiment examining clinical- skills performance after removal of grade stakes from an early objective structured clinical examination
Year: 2026
Type: preprint
Study design: Observational Study
Abstract: Background Early formative (ungraded) objective structured clinical examinations (OSCEs) are increasingly used to reduce novice-learner stress and promote feedback. However, measurement theory holds that scores on low-stakes assessments reflect not only ability but also test-taking effort, a concern rarely quantified in medical education. We examined the effect of removing grade stakes from an early clinical-skills OSCE and whether performance recovered when stakes returned. Methods We conducted an observational natural experiment at an offshore medical school. One first-semester cohort (Summer 2026, n = 241) completed the week-6 OSCE as an ungraded formative assessment; three preceding cohorts had completed the same examination summatively. All cohorts subsequently completed a graded week-12 OSCE. Both examinations included an interview station assessing aligned history-taking and communication domains. Blood-pressure measurement and vital-sign interpretation were assessed using identical criteria at both examinations and provided a content-controlled within-student anchor. Analyses used Welch’s t-tests with Mann–Whitney U confirmation, Cohen’s d, distributional diagnostics, and paired within-student comparisons. Results Under formative conditions, performance was lower than in each of the three preceding summative cohorts on both the interview station (7.9–10.3 percentage points) and physical-examination station (9.8–13.7 points) (Welch t, all p < .001; Cohen’s d 0.59–0.86). The decline was distribution-wide rather than driven by outliers and was evident across academic-performance levels: even the strongest written-examination quartile scored below prior cohorts. When stakes returned at the graded OSCE, the same students showed substantial improvement: interview performance increased by a mean of 9.6 points within students (paired d = 0.91, p < .001), and the identical blood-pressure/vital-signs anchor rose 5.7 points (paired d = 0.36, p < .001). Compared with historical week-12 cohorts, recovery was near-complete (residual deficits 2.0–5.6 points; small effects). Conclusions Transitioning an early OSCE from summative to formative assessment was associated with a large, cohort-wide decline in clinical-skills performance, followed by substantial within-student recovery when stakes returned. The magnitude, breadth, and reversibility of these findings are consistent with assessment stakes – and potentially test-taking effort – being an important contributor to observed OSCE performance. Formative OSCE scores should therefore be interpreted cautiously when used to infer clinical competence or identify students for remediation. Trial registration: Not applicable.
Source: Research Square
Authors: Tetiana Hliebova, Rakesh Calton
Institution: Ross University School of Medicine
Language: English
Subfield: Education
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7236434478
