Navigating the Political Compass: Evaluating Multilingual LLMs across Languages and Nationalities
Year: 2025
Type: conference-paper
Abstract: Large Language Models (LLMs) have become ubiquitous in today's technological landscape, boasting a plethora of applications and even endangering human jobs in complex and creative fields.One such field is journalism: LLMs are being used for summarization, generation, and even fact-checking.However, in today's political landscape, LLMs could accentuate tensions if they exhibit political bias.In this work, we evaluate the political bias of the 15 most-used multilingual LLMs via the Political Compass Test.We test different scenarios, where we vary the language of the prompt while also assigning a nationality to the model.We evaluate models on the 50 most populous countries and their official languages.Our results indicate that language has a strong influence on the political ideology displayed by a model.In addition, smaller models tend to display a more stable political ideology, i.e. ideology that is less affected by variations in the prompt.as the judge? a study on judgement bias.In Proceedings of the 2024
Authors: Chadi Helwe, Oana Balalau, Davide Ceolin
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: gold
OpenAlex ID: W4412887974