Think Before You Speak: Using Self-talk to Generate Implicit Commonsense Knowledge for Response Generation
Year: 2021
Type: preprint
Abstract: Implicit knowledge, such as common sense, is key to fluid humanconversations. Current neural response generation (RG) models are trained
end-to-end, omitting unstated implicit knowledge. In this paper, we present a
self-talk approach that first generates the implicit commonsense knowledge and
then generates response by referencing the externalized knowledge, all using
one generative model. We analyze different choices to collect knowledge-aligned
dialogues, represent implicit knowledge, and elicit knowledge and responses. We
introduce three evaluation aspects: knowledge quality, knowledge-response
connection, and response quality and perform extensive human evaluations. Our
experimental results show that compared with end-to-end RG models, self-talk
models that externalize the knowledge grounding process by explicitly
generating implicit knowledge also produce responses that are more informative,
specific, and follow common sense. We also find via human evaluation that
self-talk models generate high-quality knowledge around 75% of the time. We
hope that our findings encourage further work on different approaches to
modeling implicit commonsense knowledge and training knowledgeable RG models.
Source: arXiv (Cornell University)
Authors: Pei Zhou, Karthik Gopalakrishnan, Behnam Hedayatnia, Seokhwan Kim, Jay Pujara, Xiang Ren, Yang Liu, Dilek Hakkani‐Tür
Institutions: University of Southern California, Amazon (United States)
Language: English
Topic: Speech and dialogue systems
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W3206389558