Modeling fairness in engineering performance-based assessment: a Many-Faceted Rasch perspective
Year: 2026
Type: article
Abstract: Performance-based assessment is central to engineering education, but fairness may be threatened by rater subjectivity and task variability. This study examined a multi-task assessment using the Many-Faceted Rasch Model (MFRM). Eight faculty raters scored 468 performances from 156 undergraduate engineering students across three tasks, generating 3,744 ratings using a five-criterion rubric. MFRM jointly estimated student ability, task difficulty, rater severity, and criterion difficulty on a common logit scale. Global and facet-level fit were acceptable. Student ability showed good separation (3.45) and reliability (0.92). Male–female differences were small and non-significant (0.21 logits; t(154) = 1.18, p = .240; d = 0.19), although the 95% confidence interval (−0.14 to 0.56) was too wide to establish practical equivalence. Laboratory Experiment was the most difficult task, while Technical Presentation was the least difficult. Rater severity varied by 2.14 logits (separation = 5.82; reliability = 0.97), indicating systematic differences in scoring stringency. Bias also emerged in Rater × Task and Rater × Criterion interactions. The five-category scale functioned adequately, with ordered thresholds. Despite acceptable measurement quality, rater severity and interaction effects remain potential fairness risks. MFRM offers a framework for auditing and strengthening the transparency, equity, and defensibility of engineering performance assessment.
Source: Cogent Education
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $1,165
OpenAlex ID: W7225385629