Year: 2025
Type: conference-paper
Abstract: Large Language Models (LLMs) have demonstrated impressive performances in tasks related to coreference resolution.However, previous studies mostly assessed LLM performance on coreference resolution with nouns and third person pronouns.This study evaluates LLM performance on coreference resolution with indexicals like I, you, here and tomorrow, which come with unique challenges due to their linguistic properties.We present the first study examining how LLMs interpret indexicals in English, releasing the English Indexical Dataset with 1600 multiple-choice questions.We evaluate pioneering LLMs, including GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, and DeepSeek V3.Our results reveal that LLMs exhibit an impressive performance with some indexicals (I), while struggling with others (you, here, tomorrow), and that syntactic cues (e.g.quotation) contribute to LLM performance with some indexicals, while they reduce performance with others.
Institution: University of Southern California
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4412944712