Work

Vocabulary Shapes Cross-Lingual Variation of Word-Order Learnability in Language Models

Year: 2026

Type: conference-paper

Abstract: Why do some languages like Czech permit free word order, while others like English do not?We address this question by pretraining transformer language models on a spectrum of synthetic word-order variants of natural languages.We observe that greater word-order irregularity consistently raises model surprisal, indicating reduced learnability.Sentence reversal, however, affects learnability only weakly.A coarse distinction of free-(e.g., Czech and Finnish) and fixed-word-order languages (e.g., English and French) does not explain crosslingual variation.Instead, the structure of the word and subword vocabulary strongly predicts the model surprisal.Overall, vocabulary structure emerges as a key driver of computational word-order learnability across languages.

Source: Proceedings of the conference - Association for Computational Linguistics. Meeting

DOI: 10.18653/v1/2026.acl-long.1510

Authors: Jonas Mayer Martins, Jaap Jumelet, Viola Priesemann, Lisa Beinborn

Institutions: University of Göttingen, University of Groningen

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: Nederlandse Organisatie voor Wetenschappelijk Onderzoek

Award: Polyglot Machines: Human-like Learning of Morphologically Rich Languages