Work

Tuning of language models in Eastern European languages on Twitter/X

Year: 2025

Type: conference-paper

Abstract: We address the problem of fine-tuning large language models (LLMs) for sentiment analysis on Twitter/X in underrepresented Eastern European languages (Czech, Slovak, Polish, and Hungarian). We study the influence of a number of experimental settings on the efficiency of fine-tuning in two groups of LLMs: transfer-learning models (BERT, BERTweet or XLM-T, the latter two pre-trained on a Twitter corpus) and popular mid-sized universal models (Llama, Mistral). We show that adapter fine-tuning with as few as ≈ 600 tweets improved scores of our universal models to the level previously reported by Twitter/X-specialised models on popular datasets, while our transfer-learning models performed worse. We also show that, despite previous successful experiments with multilingual models, translating from underrepresented languages into English still improves the results of all models tested. Several other factors that influence the success of fine-tuning are also included in the study.

Source: Zenodo (CERN European Organization for Nuclear Research)

DOI: 10.5281/zenodo.17609597

Authors: Filip, Tomáš, Pavlíček, Martin, Sosik, Petr

Institutions: University of Ostrava, Silesian University in Opava

Language: English

Topic: Sentiment Analysis and Opinion Mining

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Ministerstvo Školství, Mládeže a Tělovýchovy, Slezská Univerzita v Opavě

Awards: G2502196055, G4113878522, G6402224719