Preregistration: Token cost of Norwegian Bokmål and Nynorsk relative to English across LLM tokenizers (confirmatory study)
Year: 2026
Type: article
Abstract: Preregistration of a confirmatory study of how many more tokens Norwegian Bokmål and Nynorsk text costs than English across LLM tokenizers (Anthropic Claude, OpenAI o200k_base and cl100k_base, Gemma 4 / Borealis 2, NorMistral-11B). Ten hypotheses on the parity premium (Petrov et al. 2023) are decided on FLORES-200 dev with a paired bootstrap and 99.5 % Bonferroni intervals; UD r2.19 test sets and Målfrid parallel text are reported descriptively. An earlier exploratory pilot on FLORES-200 devtest (tag pilot-v1) and the locked analysis code (signed tag prereg-v1) are included as git archives with OpenTimestamps proofs.
Author: Botten Nicolaysen, Eirik
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7226019405
