Validation-Selected Linear Interpolation Between Qwen2.5-0.5B Base and Instruct: Does It Beat Both Parents on ARC, HellaSwag and WinoGrande?
Year: 2026
Type: preprint
Abstract: Interpolating the weights of a pretrained model and its fine-tuned sibling has beaten both endpoints in some vision settings. We ask whether the same holds for a small language model. We interpolate linearly between Qwen2.5-0.5B and Qwen2.5-0.5B-Instruct on eleven coefficients, choose t^* on 300 validation items per task, and report it on 3,967 disjoint test items from ARC-Easy, ARC-Challenge, HellaSwag and WinoGrande. The pre-registered criterion was a paired 95% bootstrap interval for the gain over the better original model lying entirely above zero. It was not met. The selected t^*=0.7 reaches 50.90% test accuracy against 50.25% for the base model, a difference of +0.65 pp (95% CI [-0.23,+1.52], p=0.149): a tie. The Spearman rank correlation between validation and test accuracy over the eleven coefficients is ρ=-0.03 (p=0.93), so the validation split carried no detectable information about the test ranking. By this selection procedure and this test set, gains above about 1.5 pp are excluded; smaller gains are neither confirmed nor excluded. One model pair and four benchmarks limit generality.Preprint. The attached archive contains the LaTeX source, the per-item evaluation caches, all analysis scripts and the model revisions needed to regenerate every number, table and figure (git tag preprints-v1; entry point run_everything.sh). Author: Dheiver Francisco Santos, ORCID 0000-0002-8599-9436. License: CC BY 4.0.
Source: Figshare
Author: Dheiver Francisco Santos
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7233606673
