Year: 2025
Type: conference-paper
Abstract: Large Language Models (LLMs) are increasingly integrated into everyday applications, yet their reliability remains inconsistent, even for seemingly simple queries. By “scaling up” and “shaping up”, these models have improved average accuracy and robustness to prompt variations, but they continue to display “difficulty discordance”: they solve harder tasks while making errors on easier ones. Moreover, they show a marked reluctance to refuse answers even when uncertain. Such behaviour deprives users of clear cues about when outputs can be trusted. This work explores strategies to enhance LLM reliability through confidence-based abstention, combining uncertainty estimation techniques with measures of question difficulty to define a model’s “safe operating area”. By ensuring that queries are either answered correctly or explicitly declined, the approach aims to enhance predictability, transparency, and user trust, while providing a framework for managing model limitations.
Author: Yael Moros Daval
Institution: Universitat Politècnica de València
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
Funder: Generalitat Valenciana
OpenAlex ID: W4415195182
FWCI:
Cites:
Cited by:
Related to: