When Your Language Model Cannot Even Do Determiners Right: Probing for Anti-Presuppositions and the Maximize Presupposition! Principle
Year: 2023
Type: conference-paper
Abstract: The increasing interest in probing the linguistic capabilities of large language models (LLMs) has long reached the area of semantics and pragmatics, including the phenomenon of presuppositions.In this study, we investigate a phenomenon that, however, has not yet been investigated, i.e., the phenomenon of antipresupposition and the principle that accounts for it, the Maximize Presupposition!principle (MP!).Through an experimental investigation using psycholinguistic data and four open-source BERT model variants, we explore how language models handle different antipresuppositions and whether they apply the MP! principle in their predictions.Further, we examine whether fine-tuning with Natural Language Inference data impacts adherence to the MP! principle.Our findings reveal that LLMs tend to replicate context-based n-grams rather than follow the MP! principle, with fine-tuning not enhancing their adherence.Notably, our results further indicate a striking difficulty of LLMs to correctly predict determiners, in relatively simple linguistic contexts.
Authors: Judith Sieker, Sina Zarrieß
Institution: Bielefeld University
Language: English
Subfield: Language and Linguistics
Field: Arts and Humanities
Domain: Social Sciences
Open Access status: gold
OpenAlex ID: W4389518362