Comparison of Ordinal Classification Frameworks: An Empirical Benchmark of Performance and Efficiency
Year: 2026
Type: article
Abstract: Purpose: This research systematically compares three ordinal classification frameworks (FH, DRM, CS) using tree-based ensemble algorithms (RF, XGB, CAT). It primarily aims to evaluate their predictive performance and computational time, addressing the relative lack of empirical guidelines for ordinal tabular data.Methods: Twelve model combinations were evaluated across twelve diverse benchmark datasets categorized by class cardinality, focusing primarily on nine ordinal models. In order to ensure statistical reliability, experiments were repeated over 30 independent trials using stratified sampling, with hyper parameter optimization via grid search. Predictive performance was assessed using the QWK metric to strictly penalize ordinal misclassifications, while computational efficiency was gauged by training time.Results: Predictive performance depended on class cardinality. Performance differences were insignificant in low-cardinality settings (3–4 classes, p=0.1434) but significant in high-cardinality settings (5–7 classes, p=0.0010), where DRM_XGB ranked best (mean rank 2.0000). Conversely, training speed differed significantly across all datasets (p<0.0001); DRM_XGB was the fastest (overall mean rank 1.0000).Conclusion: The findings highlight that selecting the optimal 'Framework-Algorithm synergy' is critical for ordinal classification. The DRM_XGB combination is highly recommended for high-cardinality settings, while either DRM or CS can be flexibly deployed in low-cardinality environments depending on computational constraints. This study clarifies the structural and computational boundaries of ordinal approaches, providing evidence-based guidelines for designing robust models for ordinal tabular datasets.
Author: Nak Il Kim
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7166560670
FWCI:
Cites:
Cited by:
Related to: