Repairing One Step in the Weights
Year: 2026
Type: report
Abstract: Qwen3.5-4B fails long multiplication at one step: taking the digit at a given place. The step is traced to one attention head's query and repaired by a rank-16 change to 256 rows of one weight matrix, checked on held-out items.
Author: Latipov, Aziz
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Keywords: calculation errors, attention heads
Open Access status: green
OpenAlex ID: W7225901865
