The Development of Methods for Coreference Resolution in Ukrainian Texts Based on Large Language Models
Year: 2025
Type: article
Abstract: "This work investigates the task of automated Coreference Resolution in Ukrainian texts – a key problem in Natural Language Processing (NLP) necessary for deep semantic analysis, information extraction, machine translation, and other applications. Extended examples of coreference are provided, emphasizing the importance of the task and the specific difficulties of solving it for the Ukrainian language, primarily due to its free word order. The Transformer architecture, which underlies modern Large Language Models (LLMs), is discussed. The key characteristics of several state-of-the-art large language models used in the research (Llama 3 family, DeepSeek R1, Gemma 3, Gemini 2.0) are described, including their architectural features, sizes, openness, and support for the Ukrainian language. Two methods for applying LLMs to coreference resolution are proposed and analyzed: a method involving binary classification of potentially coreferent mention pairs, and a method, which uses the generation of a complete coreference cluster in a single query to the language model. Detailed examples of prompts to the large language model are provided for both methods. The processing of the model responses and forming the final clusters is described. A description of the coreference resolution evaluation metrics – MUC and B-cubed – is provided, including the formulas for calculating precision, recall, and F1-score for each metric. Experimental results from testing both methods on a corpus of Ukrainian texts using the specified large language models are presented and analyzed. The results demonstrate the high effectiveness of large language models for solving the coreference resolution task in Ukrainian language. A comparative analysis of the methods confirms the advantage of the cluster generation method due to lower computational costs (fewer queries) required to find coreferent mentions in the text, while the analysis of the models indicates a positive impact of reasoning capabilities (e.g., Gemini 2.0 Thinking and DeepSeek R1) on the quality of task resolution."
Authors: S. POGORILYY, P.V. BILETSKY
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4411246344