Year: 2026
Type: conference-paper
Abstract: We examine how the capabilities of large language models (LLMs) have evolved on eight Belarusian language tasks contributed in 2023 to OpenAI's Evals framework.We evaluate state-of-the-art models both on the original development sets and newly created test sets.Results demonstrate significant but non-uniform progress over this period: some tasks are almost saturated, while others show minor improvement beyond trivial baselines.Error analysis shows that certain challenges haven't yet been addressed, e.g.misidentification of non-words as legitimate vocabulary, or conversion from modern to classical orthography.We release the datasets and the generated completions.
Institution: Brno University of Technology
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7140106106