Work

Uncovering Autoregressive LLM Knowledge of Thematic Fit in Event Representation

Year: 2026

Type: conference-paper

Abstract: The thematic fit estimation task measures semantic arguments' compatibility with a given semantic role for a given predicate.We investigate if autoregressive LLMs have consistent, expressible knowledge of event arguments' thematic fit by experimenting with various prompt designs, manipulating input context, reasoning, and output forms.We set a new state-of-theart on thematic fit benchmarks, but show that closed and open weight LLMs respond differently to our prompting strategies: Closed models achieve better scores overall and benefit from multi-step reasoning, but they perform worse at filtering out generated sentences incompatible with the given predicate, role, and argument.Our analysis shows that lemma tuple input and sentence input result in surprisingly different thematic fit score distributions.

Source: Conference on Computational Natural Language Learning

DOI: 10.18653/v1/2026.conll-main.11

Authors: Safeyah Khaled Alshemali, Daniel Bauer, Yuval Marton

Institutions: Imperial College London, Columbia University, University of Washington

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: Imperial College London

OpenAlex ID: W4404088460