Work

A Many-facet RASCH Measurement Analysis to Explore Rater Effects and Rater Training in Medical School Admissions

Year: 2016
Type: dissertation
Abstract: For many undergraduate medical education (UME) programs, ratings of applicantsâ performance in interviews are integral to the decision making process. This study used many-facet Rasch measurement (MFRM) to explore the severity and consistency of 200 interviewersâ ratings of 599 applicants to a UME program. The raters varied widely in their severity and many of the raters had poor fit to the MFRM model. Although raters at four stations rated both station-specific and across-station competencies, the highest correlations were within station; the correlations across stations for the common competencies were low. No significant relationships were found between ratersâ training mode(s) (online and in-person) and rating behaviours. The findings from this work demonstrate the need to make explicit how differences in rater judgement affect fairness of applicant assessment to UME programs.
Language: English
Subfield: Education
Open Access status: green
OpenAlex ID: W2587690098
Cites: 50
Cited by: 3
Related to: 10