Work

Dialogue Act-Aided Backchannel Prediction Using Multi-Task Learning

Year: 2023

Type: conference-paper

Abstract: Produced in the form of small injections such as "Yeah!" or "Uh-Huh" by listeners in a conversation, supportive verbal feedback (i.e., backchanneling) is essential for natural dialogue.Highlighting its tight relation to speaker intent and utterance type, we propose a multitask learning approach that learns textual representations for the task of backchannel prediction in tandem with dialogue act classification.We demonstrate the effectiveness of our approach by improving the prediction of specific backchannels like "Yeah" or "Really?" by up to 2.0% in F1.Additionally, whereas previous models relied on well-established methods to extract audio features, we further pre-train the audio encoder in a self-supervised fashion using voice activity projection.This leads to additional gains of 1.4% in weighted F1.

Source: Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)

DOI: 10.18653/v1/2023.findings-emnlp.1006

Authors: Wencke Liermann, Yo-Han Park, Yong-Seok Choi, Kong Lee

Institution: Chungnam National University

Language: English

Topic: Speech and dialogue systems

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: Chungnam National University, Ministry of Science and ICT, South Korea, Institute for Information and Communications Technology Promotion

Awards: G1628181847, G4700831490, G4939975072, G6789273066