Work

Do Language Models Say "I Don't Know What You Mean"? Evaluating the Reliability of LLM-Generated Explanations Under Ambiguous and Incomplete Prompts

Year: 2026
Type: preprint
Abstract: Large Language Models (LLMs) explain things fluently, and that fluency is exactly the problem: a smooth,confident answer gives the reader very little reason to doubt it. When a question is ambiguous (it can be read inmore than one way) or incomplete (it leaves out facts needed for a specific answer), a model can quietly chooseone reading or invent the missing details, and the user may never find out. This paper reports a small, matchedpilot study of that behaviour in ChatGPT and Google Gemini. We wrote five base tasks across five domains,expressed each as a clear, an ambiguous and an incomplete prompt (15 prompts), and submitted all of them toboth models in fresh chats, giving 30 responses. Each response was rated on factual support, relevance,completeness, consistency and uncertainty handling, and was also coded for unsupported assumptions and forclarification behaviour. The ratings reported here come from a single AI-assisted evaluator and are preliminary;independent human ratings and inter-rater agreement are still pending. Within those limits, the pattern isconsistent. Reliability fell for both models once the prompt was no longer clear. On ambiguous prompts, neithermodel flagged the ambiguity in any of the ten responses, and on incomplete prompts clarification requests wererare (1 of 10 responses). We offer the prompt set, rubric and scoring formulas as a repeatable method for largerstudies.Keywords: large language models, explanation reliability, ambiguous prompts, incomplete prompts,unsupported assumptions, clarification behaviour, human evaluation
Institution: JSPM University
Language: English
Open Access status: green
OpenAlex ID: W7236761475
Cites:
Cited by:
Related to: 10