Enhancing the defensibility of cut scores: A multi-source evaluation of a modified Angoff standard-setting process in a high-stakes medical examination
Year: 2026
Type: article
Study design: Observational Study
Abstract: Introduction: Standard setting is essential in health professions education, yet establishing credible and defensible cut-off scores remains challenging, particularly when post-standard-setting evaluation is limited. Objective: To apply a systematic, multi-source approach to enhance the validity and defensibility of cut-off scores derived through a modified Angoff procedure in a high-stakes medical examination. Methods: A cross-sectional study was conducted using a final-year MBBS exit examination comprising a Multiple-Choice Question (MCQ) test and an Objective Structured Clinical Examination (OSCE). Fifteen experienced faculty applied the Modified Angoff method to determine initial cut scores. Validity evidence was collected through pre- and post-judgment surveys, psychometric analyses, and acceptability surveys involving students, judges, and medical education experts. Descriptive statistics, standard error adjustments, and independent t-tests were used. Results: Initial cut scores were 64.44/100 (MCQ) and 12.85/20 (OSCE). Judges showed high clarity, confidence, and agreement. Psychometric analysis revealed limited score variability, indicating consistent performance. Cut scores were adjusted upward by one standard error, a theoretically justified approach intended to reduce potential false-positive classifications. Acceptability was high among judges and students, with no significant group differences. Conclusion: Post standard setting evaluation added validity evidence, reinforcing the credibility and acceptability of cut off scores in high stakes examinations.
DOI: 10.3205/zma001889
Institution: University of Global Health Equity
Language: English
Subfield: Education
Field: Social Sciences
Domain: Social Sciences
Open Access status: diamond
OpenAlex ID: W7234431781
