Arithmetic Without Approximation: An Exact-Intervention Account of Number Representations in Open Language Models
Year: 2026
Type: preprint
Abstract: How does a transformer compute a+b? An exact-intervention mechanistic account on three open checkpoints (Llama-3.1-8B, Pythia-6.9B, GPT-J-6B), scoring every intervention on the model's own emitted digit. The answer position carries two orthogonal number-derived subspaces with dissociated causal roles: X_contextual, the answer read-out, along which a write steers the emitted digit; and X_number, which carries the operands and is settable while an operand is being read but read-only once it is consumed, necessary to the computation without being where the answer is set. Two tidy pictures fail: representation is not operation, and no single helix carries the computation. Four measurement choices, the causal object, the null, the dependent-variable form, and the measurement layer, each flip a conclusion, and the same discipline exposes an apparent fourth-model result as an artifact. The manuscript PDF is a separate file on this record. The attached reproducibility archive contains all producer and figure scripts, the per-cell run data, a claims ledger mapping every headline number to its producing script (including all retractions), a run-JSON schema, tested dependency versions, and a checksum manifest. Code is licensed MIT; text and figures CC-BY-4.0.
Author: Cobbins, Rob
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7204166209
Cites:
Cited by:
Related to: