Work

Repairing One Step in the Weights

Year: 2026
Type: report
Abstract: Qwen3.5-4B fails long multiplication at one step: taking the digit at a given place. The step is traced to one attention head's query and repaired by a rank-16 change to 256 rows of one weight matrix, checked on held-out items.
Language: English
Open Access status: green
OpenAlex ID: W7225901865
Cites:
Cited by:
Related to: 10