Eight Retractions in Twenty-Four Hours: A Log of What Broke While Measuring Small Language Models on One Home PC
Year: 2026
Type: article
Abstract: Over roughly twenty-four hours of continuous measurement on a single consumer GPU, eight claims were retracted. Each retraction was recorded at the moment it died, in a fixed schema that includes a field which cannot be reconstructed afterwards: why it looked true. Three of the eight were caused by the author's own tools reporting events that had not happened — a reproduction script that printed a context-overflow rejection for a request it never sent, a static checker that raised nine false findings, and an aggregator that reported zero paired cells because its stratification key included the model name. Three were framing reversals, in which changing how a task was posed reversed the result. Two were directions asserted from single-digit samples. The argument is narrow: at this scale the researcher is also the instrument-maker, and tool-generated non-events are a failure mode of the same order as statistical error, yet they are not usually reported. The log is published rather than the lessons, because the lessons are short and the log is what makes them believable. Version 1 contains cases A through H. Later versions add cases; the case identifiers are stable and will not be reassigned. The machine-readable ledger is included alongside the text.
Author: Akiyama, Satoshi
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7203923172
FWCI:
Cites:
Cited by:
Related to: