Work

Generalizability and Genre Effects in Discourse Understanding and Parsing in Rhetorical Structure Theory

Year: 2024
Type: dissertation
Abstract: Recent advances in discourse parsing create the impression that, like other types of Natural Language Processing analyses, state-of-the-art performance for high-resource languages such as English is finally becoming reliable and applicable to arbitrary data.However, there is by now substantial evidence showing that even for English, hierarchical discourse parsing within formal frameworks such as Rhetorical Structure Theory (RST) remains challenging, and that our understanding of the reasons for this are as yet limited.This dissertation aims to thoroughly investigate the impact of data diversity on RST parsing stability in English.We show that state-of-the-art neural models trained on the standard English newswire benchmark do not generalize well, even within the news domain.We also pinpoint and systematically evaluate the role of linguistic features and other factors that hinder state-of-the-art tools from achieving robust and stable performance across diverse text types and domains, through statistical methods and predictive models.Our results demonstrate that a heterogeneous training regime is critical for stable and generalizable models, across parser architectures.In addition, we show that as in shallow discourse parsing, the explicit/implicit distinction plays a role, but that long-distance dependencies are the main challenge, while lack of lexical overlap is less of a problem, at least for in-domain parsing.Moreover, this dissertation presents a corpus-based analysis of genre variation in RST discourse relations across 20 genres from two English RST corpora.The goal is to better understand genre effects at the discourse level both quantitatively and qualitatively.iii Lastly, this dissertation evaluates how effective RST extracts such as central discourse unit are as summaries by comparing them with both model-generated and human-written summaries across genres.
Author: Liu, Yang
Language: English
Open Access status: green
OpenAlex ID: W6991302565
Cites: 161
Cited by: 1
Related to: 10