Measuring Relative Grading Positions: A Network Framework and the Limits of External Validation
Year: 2026
Type: preprint
Abstract: Course grades are compared across courses even though most courses lack a common external assessment. Networks formed by overlapping student schedules can place courses and course–teacher units on a relative grading scale, but agreement with an external benchmark does not establish accuracy. This paper applies an established additive model of student and course–teacher-unit effects, decomposing the fitted unit effects into course Relative Grading Position Index (RGPI) and teacher-within-course position component (TWCP). Its validation framework distinguishes internal reproducibility, separately estimated convergence, concurrent concordance, and outcome corroboration. An exact coefficient-change identity isolates the adjustment associated with an added benchmark. In one anonymized U.S. public high school (8,192 enrollments, 229 courses, and 310 course–teacher units), course RGPI replicates across disjoint students at median r = .947 and TWCP at .701. Across 23 Advanced Placement (AP) exams, prior-year benchmark discrepancies correlate .462 with current RGPI (Fisher 95% interval .061 to .734). At the teacher level, AP-adjusted and other-course-grade-adjusted calibrations yield similar median agreement with TWCP across disjoint students (.727 and .731). Adding AP to a network-based grade baseline increases median disjoint agreement by .023; the same-student slope ratio is 1.069, with 2.5th–97.5th bootstrap percentiles 0.947 to 1.132. Simulations with known grading offsets show that higher agreement and steeper calibrations can accompany either improved or poorer recovery, including when accuracy and agreement use the same half-sample estimates and when grades are uncapped. The evidence supports diagnostic inquiry on supported, persistent patterns. Grade correction requires an independently justified target and further validation.
Source: EdArXiv (OSF Preprints)
Author: Yi Yang
Language: English
Field: Decision Sciences
Domain: Social Sciences
SDG: Quality education
Open Access status: green
OpenAlex ID: W7225458764