Work

Cross-Family Transfer of Sparse Autoencoder Features Is Real but Readout-Dependent: A Preregistered Reanalysis

Year: 2026
Type: preprint
Abstract: Do sparse autoencoder (SAE) features matched between language models of different families act on the second model the way that model's own feature does? A common shortcut maps a source decoder direction into the target residual stream, adds it, and reads the target SAE encoder at the same tensor. We show that this readout is a first-order encoder projection (Spearman 0.97 with the encoder-row projection in a GPT-2 to Pythia-70m pilot), that the pilot's "130 times random" headline rested on a near-zero denominator, and that its negative Procrustes R² (−1.15) was a scale artifact (0.49 with a fitted scale). We then run a preregistered study on GPT-2 small, Pythia-70m and Qwen3-1.7B-Base with public SAEs, tokenizer-independent word-level matching against a multiple-comparisons-aware null, and readouts that pass through the target model: the downstream SAE code and the next-token distribution. Transfer exceeds mismatched and Gaussian nulls in all four directed pairs (all 24 primary tests survive Holm correction). But raw downstream readouts also credit large nonspecific effects: a mismatched mapped direction reaches τL between 0.46 and 0.64 of the native effect at the preregistered magnitude. Net of that null, transfer is 0.17 to 0.42 of native for the downstream SAE readout. Contrary to our preregistered expectation, the encoder readout does not overstate transfer; it is below the raw causal readouts in every pair. Transfer is predicted mainly by the cosine between mapped and native decoder directions (pooled ρ = 0.63), and the pilot's match-correlation predictor does not replicate.
Language: English
Open Access status: green
OpenAlex ID: W7234852532
Cites:
Cited by:
Related to: 10