Work

Role of Context in Unsupervised Sentence Representation Learning: the Case of Dialog Act Modeling

Year: 2023

Type: conference-paper

Abstract: Unsupervised learning of word representations involves capturing the contextual information surrounding word occurrences, which can be grounded in the observation that word form is largely disconnected from word meaning.While there are fewer reasons to believe that the same holds for sentences, learning through context has been carried over to learning representations of word sequences.However, this work pays minimal to no attention to the role of context in inferring sentence representations.In this article, we present a dialog act tag probing task designed to explicitly compare content-, and context-oriented sentence representations inferred on utterances of telephone conversations (SwDA).Our results suggest that there is no clear benefit of context-based sentence representations over content-based sentence representations.However, there is a very clear benefit of increasing the dimensionality of the sentence vectors in nearly all approaches.

Source: Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)

DOI: 10.18653/v1/2023.findings-emnlp.588

Authors: Rastislav Hronsky, Emmanuel Keuleers

Institution: Tilburg University

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

OpenAlex ID: W4389520430

FWCI:

Cites: 40

Cited by:

Related to: 10