Year: 2024
Type: preprint
Abstract: In this paper, we investigate the problem of "generation supervision" in large language models, and present a novel bicameral architecture to separate supervision signals from their core capability, helpfulness. Doppelgänger, a new module parallel to the underlying language model, supervises the generation of each token, and learns to concurrently predict the supervision score(s) of the sequences up to and including each token. In this work, we present the theoretical findings, and leave the report on experimental results to a forthcoming publication.
Source: arXiv (Cornell University)
Authors: Ghasemlou, Shervin, Katiyar, Ashish, Saraf, Aparajita, Moon, Seungwhan, Pujari, Mangesh, Donmez, Pinar, Damavandi, Babak, Kumar, Anuj
Institution: Meta (United States)
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4403592964