Classifier calibration: a survey on how to assess and improve predicted class probabilities
Year: 2023
Type: article
Abstract: This paper provides both an introduction to and a detailed overview of the principles and practice of classifier calibration. A well-calibrated classifier correctly quantifies the level of uncertainty or confidence associated with its instance-wise predictions. This is essential for critical applications, optimal decision making, cost-sensitive classification, and for some types of context change. Calibration research has a rich history which predates the birth of machine learning as an academic field by decades. However, a recent increase in the interest on calibration has led to new methods and the extension from binary to the multiclass setting. The space of options and issues to consider is large, and navigating it requires the right set of concepts and tools. We provide both introductory material and up-to-date technical details of the main concepts and methods, including proper scoring rules and other evaluation metrics, visualisation approaches, a comprehensive account of post-hoc calibration methods for binary and multiclass classification, and several advanced topics.
Source: Machine Learning
Authors: Telmo Silva Filho(Conceptualization, Formal analysis, Methodology, Software, Writing – original draft, Writing – review & editing), Hao Song(Conceptualization, Formal analysis, Methodology, Software, Writing – original draft, Writing – review & editing), Miquel Perello-Nieto(Formal analysis, Methodology, Software, Visualization, Writing – original draft, Writing – review & editing), Raul Santos-Rodriguez(Formal analysis, Writing – review & editing), Meelis Kull(Conceptualization, Formal analysis, Methodology, Writing – review & editing), Peter Flach(Conceptualization)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $2,990
Awards: G1309768829, G1633672591, Contextual uncertainty and representation learning in machine perception, SPHERE - A Sensor Platform for HEalthcare in a Residential Environment (IRC Next Steps), G3814478501, G3855435662, Turing AI Fellowship: Interactive Annotations in AI, The Alan Turing Institute, G7935266708, G8581904048, G8597009950
OpenAlex ID: W4376891107
