Optimizing statistical evaluation of multiclass classification in diagnostic radiology: a study of the two-parameter multidimensional nominal response model
Year: 2024
Type: article
Abstract: Purpose: This study aimed to enhance the multidimensional nominal response model (MDNRM) for multiclass classification in diagnostic radiology. Materials and Methods: This retrospective study involved the extension of the conventional nominal response model (NRM) to create the two-parameter MDNRM (2PL-MDNRM). Seven models of MDNRM, including the original MDNRM and subtypes of 2PL-MDNRM, were employed to estimate test-takers' abilities and test item complexity. These models were applied to a clinical diagnostic radiology dataset. Rhat values were calculated to evaluate model convergence. Additionally, values of the widely applicable information criterion (wAIC) and Pareto-smoothed importance sampling leave-one-out cross-validation (LOO) were calculated to evaluate the goodness of fit of the seven models. The best-performing model was selected based on the values of wAIC and LOO. Probability of direction (PD) was used to evaluate whether one estimated parameter significantly differed. Results: using the truncated normal distribution. Notably, one test-taker (radiologist) exhibited significantly superior ability compared to another based on PD results from the best models, while no significant difference was observed in nonoptimal models. Conclusion: 2PL-MDNRM successfully achieved parameter estimation convergence, and its superiority over the original MDNRM was demonstrated through wAIC and LOO values.
Source: PeerJ Computer Science
Authors: Mizuho Nishio, Eiji Ota
Institutions: Kobe University, Futaba (Japan)
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $1,395
OpenAlex ID: W4403140063