Clinically Relevant Measures of Fit? A Note of Caution
Year: 2012
Type: article
Abstract: Risk reclassification methods have become popular in the medical literature as a means of comparing risk prediction models. In this issue of the Journal, Pencina et al. (Am J Epidemiol. 2012;176(6):492-494) present further results for continuous measures of model discrimination and describe their characteristics in nested models with normally distributed variables. Measures include the change in the area under the receiver operating characteristic curve, the integrated discrimination improvement, and the continuous net reclassification improvement. Although theoretically interesting, these continuous measures may not be the most appropriate to assess clinical utility. The continuous net reclassification improvement, in particular, is a measure of effect rather than model improvement and can sometimes exhibit erratic behavior, as illustrated in 2 examples. Caution is needed before using this as a measure of improvement. Further, the test of the continuous net reclassification improvement and that for the integrated discrimination improvement are similar to the likelihood ratio test in nested models and may be overinterpreted. Reclassification in risk strata, while requiring thresholds, may be more relevant clinically with its ability to examine potential changes in treatment decisions.
Source: American Journal of Epidemiology
DOI: 10.1093/aje/kws208
Author: N. R. Cook
Institutions: Brigham and Women's Hospital, Harvard University
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2130812680
