Get started
Work

Uncertainty quantification in ordinal classification: A comparison of measures

Year: 2025
Type: article
Abstract: Uncertainty quantification has received increasing attention in machine learning in the recent past, but the focus has mostly been on standard (nominal) classification and regression so far. In this paper, we address the question of how to quantify uncertainty in ordinal classification, where class labels have a natural (linear) order. We reckon that commonly used measures such as Shannon entropy or variance are not appropriate for the ordinal case. In our search for better measures, we draw inspiration from the social sciences literature, which offers various measures to assess so-called consensus or agreement in ordinal data. We argue that these measure, or, more specifically, the dual measures of dispersion or polarization, do have properties that qualify them as measures of uncertainty. Furthermore, inspired by binary decomposition techniques for multi-class classification in machine learning, we propose a new method that allows for turning any uncertainty measure into an ordinal uncertainty measure in a generic way.We evaluate all measures in an empirical study on twenty-three ordinal benchmark datasets, as well as in a real-world case study on automotive goodwill assessment. Our studies confirm that that dispersion measures and our binary decomposition method surpass conventional (nominal) uncertainty measures.
DOI: 10.1016/j.ijar.2025.109479
Language: English
Open Access status: hybrid
Estimated APC paid: $3,050
OpenAlex ID: W4410815531
FWCI: 2.062
Cites: 58
Cited by: 5
Related to: 10