Can Small Language Models Distinguish Rusyn, Belarusian, and Ukrainian? An Empirical Study of Hallucination and Subsumption in Local 4B Language Models
Year: 2026
Type: preprint
Abstract: Quantised 4B-parameter LLMs are usually marketed as an ideal choice for local tasks because of their multilinguality; however, some languages still remain poorly understood. In this research, three 4-bit (Q4_K_M) models are evaluated: Qwen3.5-4B, Ministral-3-3B-Instruct, and Nemotron-3.5-Bolt-E4B in four configurations on zero-shot language identification with proofs for Rusyn, Belarusian, and Ukrainian (N = 15 parallel sentences; 4 configurations × 15 sentences = 60 deterministic runs). Results show Ministral invents grammatical rules to justify languages wrongly labelled as Russian, while Nemotron scores 5/5 on both Ukrainian and Belarusian, but 0/5 on Rusyn by mistaking its reflexive particle for Belarusian. Finally, enabling chain-of-thought (<think>) in Qwen3.5-4B triggers unclosed reasoning and repetition loops (0% compliance, 69.80s), whereas disabling it (--reasoning-budget 0) achieves 86.7% accuracy (80% on Rusyn) in 9.60s. Code, dataset, and logs: https://github.com/VladyslavKlochko/east-slavic-lid-4b
Author: Klochko, Vladyslav
Institution: Varna University of Management
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7236047854
