Improving Accuracy and Calibration of Deep Image Classifiers With Agreement-Driven Dynamic Ensemble
Year: 2024
Type: article
Abstract: One of the biggest challenges when considering the applicability of Deep Learning systems to real-world problems is the possibility of failure incriticalsituations. Possible strategies to tackle this problem are two-fold: (i) models need to be highly accurate, consequently reducing this risk of failure; (ii) facing the impossibility of completely eliminating the risk of error, the models should be able to inform the level of uncertainty at the prediction level. As such, state-of-the-art DL models should beaccurateand alsocalibrated, meaning that each prediction has to codify its confidence/uncertainty in a way that approximates the true likelihood of correctness. Nonetheless, relevant literature shows that improvements inaccuracyandcalibrationare not usually related. This motivates the development of Agreement-Driven Dynamic Ensemble, a deep ensemble method that - by dynamically combining the advantages of two different ensemble strategies - is capable of achieving the highest possible accuracy values while obtaining also substantial improvements in calibration. The merits of the proposed algorithm are shown through a series of representative experiments, leveraging two different neural network architectures and three different datasets against multiple state-of-the-art baselines.
Authors: Pedro Conde, Rui L. Lopes, Cristiano Premebida
Institutions: University of Coimbra, Critical Software (Portugal)
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,995
OpenAlex ID: W4405520473