Year: 2026
Type: conference-paper
Abstract: Large language models (LLMs) often respond even when prompts omit critical details or include misleading information, leading to hallucinations or reinforced misconceptions.We study how to evaluate and improve LLMs' ability to decide when and what to ask for clarification without sacrificing task performance.We introduce AskBench, an interactive benchmark that converts standard QA pairs into multi-turn interactions with explicit checkpoints.A unified judge loop evaluates final answers and simulates user responses as needed.AskBench covers two settings: AskMind, with intent-deficient queries requiring clarification, and AskOverconfidence, with queries containing false premises that must be identified and corrected.We further propose rubric-guided reinforcement learning with verifier-based rewards (RLVR), which uses structured rubrics to encourage targeted clarification.Experiments show consistent improvements in accuracy, rubric adherence, and interaction efficiency, with strong generalization to unseen domains.
Authors: Jiale Zhao, Ke Fang, Lu Cheng
Institution: University of Illinois Chicago
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: Collaborative Research: III: Medium: Knowledge discovery from highly heterogeneous, sparse and private data in biomedical informatics, SCH: Fair and Interpretable Prediction of Alzheimer's Disease and MCI using LLMs, G6671297155, Collaborative Research: FDT-BioTech: Advancing Alzheimer's Disease Understanding and Treatments through Digital Twin Modeling, CAREER: Conformal Methods for Responsible Language Models
OpenAlex ID: W7166870529