Artificial Intelligence Predictions And Human Perception: A Comparison Of User Confidence And Accuracy
Year: 2026
Type: conference-paper
Abstract: This study aims to examine the relationship between the technical predictive accuracy of artificial intelligence (AI) models and the confidence that expert users place in these predictions. Whilst the success of AI systems is currently measured predominantly by technical metrics, the role of user confidence in this process and the ‘accuracy-confidence calibration’ are frequently overlooked. In this context, the research analyses how AI outputs are perceived by users and to what extent this perception aligns with objective model accuracy. Data was collected from 50 academics as part of the study. Participants were presented with 20 different prediction examples from an AI model and asked to rate their confidence levels on a scale of 1 to 5. Descriptive statistics, along with an independent samples t-test and correlation analysis, were used in the data analysis. The findings indicate a positive, significant, yet moderate relationship between AI prediction accuracy and user confidence. A notable finding is that some low-accuracy predictions received high confidence scores, whilst some high-accuracy predictions were met with relatively low confidence. This situation highlights that user confidence is sometimes miscalibrated and that ‘Algorithm Aversion’—as described in the literature—along with cognitive biases, plays a role in decision-making processes. The findings also support the view that confidence is closely linked not only to accuracy but also to factors such as the system’s explainability and presentation style, and that Explainable Artificial Intelligence (XAI) approaches play a critical role in this regard. The study concludes that, when evaluating AI systems, user-centered approaches are necessary beyond technical performance. It is recommended that AI interfaces be designed transparently and that user literacy be enhanced. For future research, it is suggested that the direct impact of XAI applications on trust be examined using experimental designs across different professional groups, and that user perceptions be analyzed in depth using qualitative methods.
Institution: Istanbul Arel University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7169651469