Work

Grafting a Machine-Checked Adder into Qwen3.5-0.8B without Gradients or Weight

Year: 2026
Type: preprint
Abstract: We write, rather than train, an exact multi-digit adder into a pretrained language model. The host is Qwen3.5-0.8B, a hybrid of Gated DeltaNet and gated full-attention layers with RMSNorm. The adder is a set of new residual coordinates, attention heads and MLP neurons whose every weight is set by hand from a written design; no gradient step is taken and no original weight value changes. It fires only on the fixed, zero-padded prompt "\n{a:0W}+{b:0W}=", with operand widths W = 10 and W = 20, and writes the W + 1 answer digits. Two proofs come with it. (a) Core correctness: the discrete algorithm computes a + b for every W (Lean); that the network realises it within margins on every input of up to 36,864 tokens rests on paper margin lemmas and a float64 numeric certificate; and the stored weights equal an independent restatement bit for bit. (b) Non-interference: in the reals the grafted logits are the base logits plus a private read-out (Lean, for a stated model of the layer under paper runtime premises), and with the gate off that read-out is exactly ±0 in fp32 (a paper lemma). On 1,000 fresh samples per operand length, the base model answers 0 of them for every d < W and 14 at d = W = 20; the grafted model answers all 20,000. With the gate off, the side-car runtime's logits are bit-identical to the base's at all 63,233,866 positions of the non-interference data, and the in-place checkpoint stays within KL 3.17 × 10^−8. A pre-registered gradient contrast did not hold: a LoRA trained at matched GPU-hours on lengths ≤ 10 reaches 1,000/1,000 at d = 11 in 2 of 3 seeds. Beyond the fixed format, at a separately pre-registered and weaker level (a second written cartridge with a free-text trigger, gated by a closed-form probe of the host's activations, still with no gradient step; no Lean part, and its off-gate exactness is measured, not proven), base generations with the cartridge's answers spliced in were more often correct on a constructed GSM8K-test set whose numbers are scaled up so that additions become harder: +0.51 pp per problem (95% CI [+0.32, +0.73]; sign-flip test p < 10^−5), with no measurable harm on GSM8K as written or on three unrelated task sets (where few generations can change); the effect is small, and its limits are stated with it. Notes:This version is released with its LaTeX source and the generated number file; numbers were generated at repository commit 1b79c7d. H1–H3 were pre-registered (prereg/PREREG_C.md, tag prereg-C-frozen, commit e44bcc5) and H4 separately (prereg/PREREG_D.md, tag prereg-D-frozen, commit 3f5de6a). The Lean development, cartridge builders, checking scripts, result files and both pre-registrations will follow as a separate artifact after a licence check of the third-party data they contain. The base model's weights are not redistributed. The construction, proofs, audits and evaluation were carried out by Claude agents (Anthropic) working in separate roles under the author's direction; the claims and their wording are the author's.
Language: English
Subfield: Software
Open Access status: green
OpenAlex ID: W7237364816
Cites:
Cited by:
Related to: