Year: 2024
Type: conference-paper
Abstract: Tailoring outputs from large language models, like ChatGPT, to implicit user preferences remains a challenge despite their impressive generative capabilities.In this paper, we propose a tri-agent generation pipeline comprising a generator, an instructor, and an editor to enhance output personalization.The generator produces an initial output, the instructor automatically generates editing instructions based on user preferences, and the editor refines the output to align with those preferences.The inferenceonly large language model (ChatGPT) serves as both the generator and editor, with a smaller model acting as the instructor to guide output generation.We train the instructor using editorsteered reinforcement learning, leveraging feedback from a large-scale editor model to optimize instruction generation.Experimental results on two abstractive summarization datasets demonstrate the effectiveness of our approach in generating outputs that better meet user expectations.1
Institutions: Microsoft (United States), University of British Columbia
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7126414014