Best-of-N: Inference-Time Compute for Small Language Models
Year: 2026
Type: article
Abstract: Technical note TR-2026-02, superseding TR-2026-01. We measure what happens when a frozen language model is sampled 128 times perproblem and the result is selected rather than taken as given. On GSM8K,Qwen2.5-0.5B-Instruct goes from 45.3% at a single sample to 66.5% undermajority voting, with no weights modified. Coverage - whether any trajectoryreaches the correct answer - reaches 93.5%. We separate the gain into the part attributable to selection (+20.2 points) andthe part attributable to simply having a trajectory that finished (+1.0points), and show that the second term is a property of the experimental setuprather than of the method: at a tighter token budget it was four times larger,and would conventionally have been reported as method gain. Selection is invariant under permutation of the trajectory pool, for everyselector other than random. This is not automatic: it requires an equivalencepartition built as a transitive closure, exactly-rounded summation of theweights, and tie-breaking that does not encode arrival order. The property isverified directly and is visible in the published output. The same protocol is applied to eight models, from SmolLM2-1.7B toQwen3.8-27B. Every one improves, by between +2.4 and +37.3 points, and thegain shrinks as the base model gets better. Coverage stays above what the votereturns on all eight, by a median of 9 points. All 25,600 trajectories are published in full, and every figure is recomputedfrom them by a script that re-extracts answers from the raw reasoning textrather than re-counting stored ones. This version supersedes TR-2026-01. The measurements in that report wereproduced by version 1.0.0 of the software, which contained defects in answerextraction and in the handling of truncated trajectories, and they are notreproducible with the corrected implementation. They should not be cited.Specifically withdrawn is the claim of a trained outcome reward model reachingROC-AUC 0.910 and a 16.6-point improvement over majority voting. Software: pip install bestofn - Apache License 2.0Code and data: https://github.com/voidlinestudios12-jpg/Interlace-AI
Author: Areces Rivera, Alejandro
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7203474926
FWCI:
Cites:
Cited by:
Related to: