Work

Retrospective Learning from Interactions

Year: 2025

Type: conference-paper

Abstract: Multi-turn interactions between large language models (LLMs) and users naturally include implicit feedback signals.If an LLM responds in an unexpected way to an instruction, the user is likely to signal it by rephrasing the request, expressing frustration, or pivoting to an alternative task.Such signals are task-independent and occupy a relatively constrained subspace of language, allowing the LLM to identify them even if it fails on the actual task.We introduce RESPECT, a method to learn from such signals in past interactions via retrospection without additional annotations.We deploy RESPECT in a new multimodal interaction scenario, where humans instruct a multimodal LLM to solve an abstract reasoning task with a combinatorial solution space.Through thousands of interactions with humans, we show how RESPECT gradually improves task completion rate from 31% to 82%, all without any external annotation.

Source: Proceedings of the conference - Association for Computational Linguistics. Meeting

DOI: 10.18653/v1/2025.acl-long.1200

Authors: Zizhao Chen, Mustafa Omer Gul, Yiwei Chen, Gloria Geng, Anne Wu, Yoav Artzi

Institutions: Cornell University, Cornell Tech

Language: English

Topic: AI-based Problem Solving and Planning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: bronze

Funders: National Science Foundation, National Aeronautics and Space Administration, University of Texas at Austin, Open Philanthropy Project

Awards: CAREER: Scalable Learning and Models for Mapping Instructions to Actions, Computation for the Endless Frontier, Category I: Crossing the Divide Between Today's Practice and Tomorrow's Science, Frameworks: arXiv as an accessible large-scale open research platform, G890701178