Who Did You Blame When Your Project Failed? Designing a Corpus for Presupposition Generation in Cross-Examination Dialogues
Year: 2024
Type: conference-paper
Abstract: This paper introduces the corpus for the novel task of presupposition generation -a natural language generation problem where a list of presuppositions carried by the given input sentence, in the context of the presented research -given the cross-examination question, is generated.For this, two datasets, PECaN (Presupposition, Entailment, Contradiction and Neutral) and PGen (Presuppostion Generation) are designed and used to fine-tune BERT (Devlin et al., 2019) and T5 (Raffel et al., 2020) models for classification and generation tasks respectively.Various corpora construction methods are proposed, ranging from performing manual annotations, prompting the GPT 3.0 model, to augmenting data from the existing corpora.The fine-tuned models achieved high accuracy on the novel Presupposition as Natural Language Inference (PNLI) task which extends the traditional Natural Language Inference (NLI) task, incorporating instances of presupposition into classification.T5 outperforms BERT by a broad margin, achieving an overall accuracy of 84.35% compared to 71.85% of BERT, and specifically when classifying presuppositions (93% accuracy of T5 vs. 73% of BERT).Regarding presupposition generation, we observed that despite the limited amount of data used for fine-tuning, the model displays an emerging proficiency in generation presuppositions reaching ROUGE scores of 43.47, adhering to systematic patterns that mirror valid strategies for presupposition generation, although failing to generate the complete lists.
Institutions: University of Groningen, Saarland University
Language: English
Topic: Speech and dialogue systems
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W7131855486