Year: 2026
Type: article
Abstract: Recent work in neural network interpretability has suggested that hidden activations of some deep models can be viewed as linear projections of much higher-dimensional vectors of sparse latent "features."In general, this kind of representation is known as a superposition code.This work presents an information-theoretic account of superposition codes in a setting applicable to interpretability.We show that when the number k of active features is very small compared to the number N of total features, simple inference methods currently used by sparse autoencoders can reliably decode a d-dimensional superposition code when d is a constant factor greater than the Shannon limit.Specifically, when ln k/ ln N ≤ η < 1 and H is the entropy of the latent vector in bits, we prove necessary and sufficient asymptotic conditions of the form d/H > C(η) for certain constants C(η).However, for a fixed η, we show empirically that the critical value of d/H depends significantly on what decoding method is used.For example, when η = 0.3, an estimation procedure based on the popular top-k activation function requires d/H > 4 dimensions per bit, while a method that requires only around 3 times the computational expense succeeds with d/H > 2. We hope this work helps connect research in interpretability with perspectives from compressive sensing and information theory.
Institution: University of Luxembourg
Language: English
Subfield: Theoretical Computer Science
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7219448161