Work

User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal

Year: 2025

Type: conference-paper

Abstract: Once language models (LMs) are deployed, they can interact with users long-term, ideally evolving based on their feedback.Asking for direct user feedback can be disruptive; thus, we study harvesting implicit user feedback from user-LM interaction logs.We study two user-LM interaction datasets (WildChat and LMSYS).First, we analyze user feedback in the user-LLM conversation logs, providing insights into when and why such feedback occurs.Second, we study harvesting learning signals from such implicit user feedback.Specifically, we study whether incorporating the contents of user feedback (e.g., user wanted clarification), in addition to the polarity of the feedback, can improve the model performance.We observe mixed results, showing this helps in short human-designed questions (MTBench) but not on longer and more complex questions (Wild-Bench).Together, we provide an in-depth study of implicit user feedback, showing its potential and limitations.

Source: Conference on Empirical Methods in Natural Language Processing (EMNLP)

DOI: 10.18653/v1/2025.emnlp-main.133

Authors: Yuhan Liu, Michael JQ Zhang, Eunsol Choi

Institution: New York University

Language: English

Topic: Speech and dialogue systems

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: National Science Foundation

Award: Collaborative Research: RI: Medium: Multilingual Long-form QA with Retrieval-Augmented Language Models