Utility of Weights for Weighted Kappa as a Measure of Interrater Agreement on Ordinal Scale
Year: 2008
Type: article
Abstract: Kappa statistics, unweighted or weighted, are widely used for assessing interrater agreement. The weights of the weighted kappa statistics in particular are defined in terms of absolute and squared distances in ratings between raters. It is proposed that those weights can be used for assessment of interrater agreements. A closed form expectations and variances of the agreement statistics referred to as AI1 and AI2, functions of absolute and squared distances in ratings between two raters, respectively, are obtained. AI1 and AI2 are compared with the weighted and unweighted kappa statistics in terms of Type I Error rate, bias, and statistical power using Monte Carlo simulations. The AI1 agreement statistic performs better than the other agreement statistics.
Author: Moonseong Heo
Institution: Albert Einstein College of Medicine
Language: English
Subfield: Statistics, Probability and Uncertainty
Field: Decision Sciences
Domain: Social Sciences
Open Access status: diamond
OpenAlex ID: W2679733293