Year: 2022
Type: conference-paper
Abstract: Weighted decoding methods composed of the pretrained language model (LM) and the controller have achieved promising results for controllable text generation.However, these models often suffer from a control strength/fluency trade-off problem as higher control strength is more likely to generate incoherent and repetitive text.In this paper, we illustrate this tradeoff is arisen by the controller imposing the target attribute on the LM at improper positions.And we propose a novel framework based on existing weighted decoding methods called CAT-PAW 1 , which introduces a lightweight regulator to adjust bias signals from the controller at different decoding positions.Experiments on positive sentiment control, topic control, and language detoxification show the effectiveness of our CAT-PAW upon 4 SOTA models 2 .
Authors: Yuxuan Gu, Xiaocheng Feng, Sicheng Ma, Jiaming Wu, Heng Gong, Bing Qin
Institutions: Harbin Institute of Technology, Peng Cheng Laboratory
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Award: G2369031756
OpenAlex ID: W4285286564