The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Convergent Category Geometry in Small Language Models
Year: 2026
Type: preprint
Abstract: Bürger et al. (2024) demonstrated that truth representations in large language modelsare universal across statement polarity but reside within a multidimensional subspace. Thismanuscript extends that framework along three questions: how the dimensionality of the subspacedepends on the model’s knowledge, which architectural component builds the truth direction,and what the direction is a mixture of. In Part I, using a training-free directional probe derivedfrom the SVD of hidden-state minimal pairs, we show that the dimensionality of truth isknowledge-dependent: the signal is highly concentrated on a single axis for behaviorally knownfacts (held-out AUC 0.938) and becomes diffuse as model knowledge decreases or materialheterogeneity increases. Seven falsification experiments sustain the one-dimensional reading onknown facts, and the same decomposition recovers the supervised polarity direction of Bürger etal. at cosine 0.959 without polarity labels. In Part II, we follow the direction across four basemodels from two families under pre-registered predictions. A relational empirical law emerges:attention modules propagate truth frames they did not write, whereas feed-forward networksoppose the frame of their current block to deposit next-frame content; the law generalizes,and corrects, a peak-anchored claim of our own exposed by a provenance control built againstour own instrument. Post-peak signal decay is causally attributed, on all four models, to theSwiGLU value stream rather than its gate. Finally, per-category truth axes form a semanticallysigned arrangement that contradicts the lexical surface and converges across families (Mantelp =0.0009), indicating that the geometry of the mixture belongs to the knowledge domain ratherthan to the architecture
Author: Vicidomini, Francesco Karim
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7168272863
Cites:
Cited by:
Related to: