Year: 2025
Type: conference-paper
Abstract: Discourse structure annotation is known to involve a high level of subjectivity, which often results in low inter-annotator agreement.In this paper, we focus on 'legitimate disagreements', by which we refer to multiple valid annotations for a text or text segment.We provide a new dataset of English and German texts, where each text comes with two parallel analyses (both done by well-trained annotators) in the framework of Rhetorical Structure Theory.Using the RST-Tace tool, we build a list of all conflicting annotation decisions and present some statistics for the corpus.Thereafter, we undertake a qualitative analysis of the disagreements and propose a typology of underlying reasons.From this we derive the need to differentiate two kinds of ambiguities in RST annotation: those that result from inherent linguistic ambiguity, and those that arise from specifications in the theory and/or the annotation schemes.
Source: Linguistic Annotation Workshop
Authors: Freya Hewett, Manfred Stede
Institution: University of Potsdam
Language: English
Subfield: Philosophy
Field: Arts and Humanities
Domain: Social Sciences
Open Access status: bronze
Funder: Deutsche Forschungsgemeinschaft
OpenAlex ID: W4412944238