Work

Weight-Space Audit of MIT-p50c and DNA Weight Editing (SGNE-CC) on Qwen3.5-0.8B at a 32,768-Token Generation Limit (EN / RU)

Year: 2026
Type: article
Abstract: Weight-Space Audit of MIT-p50c and DNA Weight Editing (SGNE-CC) on Qwen3.5-0.8B at a 32,768-Token Generation Limit (EN / RU) This archival deposit provides the bilingual (English and Russian) benchmark paper (01, 02), standalone BF16 checkpoint builder and FP64 weight-geometry verifier (03_apply_dna_patch.py), compact BF16 weight patch (04_sgne_cc_dna_delta.npz, 3.13 MiB, SHA-256: 4e4d36e032869bc10f56e2a2033a52d3f2774321cf867ef6f74274a7c0d1566d), and complete reproducibility repository archive (05_SGNE_CC_Complete_Repository_Suites_Traces_and_Charts.zip, 68 files including 20 charts, evaluation suites, offline CPU verifiers, and 26 raw generation trace files across 37 SHA-256-verified artifacts). Weight-Space Geometry Audit: In MIT-p50c (Sparks of Weight-Space Programming, Takehi et al., 2026), 128 feed-forward mlp.down_proj output columns are modified across 2 layers (indices 15 and 19), with the first singular mode accounting for over 99.999% of update energy and individual column Euclidean norms increasing by up to +114.2614%. In SGNE-CC (Spherical Gated Norm-preserving Edit, Column-Constrained, model Qwen3.5-0.8B-SGNE-CC), 1,984 output columns (802 mlp.down_proj columns across 15 feed-forward layers and 1,182 linear_attn.out_proj columns across 9 recurrent GatedDeltaNet layers, leaving all 6 attention layers unchanged) are rotated on their Euclidean spheres without gradient retraining while preserving individual column norms within 0.1625% after BF16 rounding (median rotation angle 8.8°, maximum 93.7°), together with 1 recurrent decay-gate row in layer 16 (2,032,640 scalars in replacement region). Empirical Summary at a 32,768-New-Token Limit: Under greedy decoding (temperature 0, no presence penalty) on the 100-problem stratified MATH-500 sample, accuracy is 3.0% (3/100) for Base Qwen/Qwen3.5-0.8B, 4.0% (4/100) for MIT-p50c, and 44.0% (44/100) for SGNE-CC (paired McNemar p = 2.84 × 10^-11). Under model-card sampling (temperature=0.6, top_p=0.95, top_k=20, presence_penalty=1.5), full MATH-500 accuracy in direct PyTorch runs is 37.6% (188/500, 20.3k mean tokens) for Base, 50.2% (251/500, 18.3k tokens) for MIT-p50c, and 47.0% (235/500, 7.4k tokens, p = 0.145) for SGNE-CC (48.1% and 7.9% in the 4-run vLLM reference for MIT-p50c and Base). Across out-of-domain and multilingual suites: MGSM-240 (25.8% / 57.1% / 41.7%), BBH-160 (1.9% / 50.6% / 41.3%), HumanEval-164 (15.9% / 31.1% / 28.0%), and MultiSpeech-64 at temperature 0 (3.1% / 26.6% / 35.9%). Methodological Scope and Licensing: Column norm preservation is measured directly on the weights; isolating its causal effect from update rank, layer selection, and the layer-16 gate row requires separate ablations, and shorter generations under model-card sampling trade off accuracy on several tasks. Author code, the weight patch, and manuscript text are licensed under Apache-2.0; bundled third-party datasets, code snippets, and MIT reference logs retain their original licenses (MIT, CC-BY-4.0, CC-BY-SA-4.0). Companion repository: https://github.com/Nondual-Observer/dna-weight-editing.
Language: English
Open Access status: green
OpenAlex ID: W7233590141
FWCI:
Cites:
Cited by:
Related to: