When (Not) to Trust AI: An Interpretable Approach to Assessing Machine Learning Predictions
Type: article
Abstract: Artificial intelligence has become pervasive across business, society, and research. In particular, the predictions of machine learning (ML) models are used for decision support in high-stakes domains such as healthcare and finance. However, ML model predictions are naturally not always correct, and false ML model predictions can have severe consequences. We propose U-EXPLORE, a novel approach for reject/trust-decisions of ML model predictions which aims to facilitate improved AI-augmented decision-making. U-EXPLORE reconstructs the decision behavior of ML models and develops a set of strongly performing ML models to exploit model multiplicity. The reject/trust-decision is based on three risk factors, thus highlighting why a prediction was rejected or trusted. An evaluation on four real-world datasets from high-stakes domains shows that U-EXPLORE provides strong decision accuracy and outperforms competing approaches. In addition, due to its interpretability, U-EXPLORE not only enhances transparency, but also can facilitate the improvement of ML models.
Institution: University of Regensburg
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7222989566