Beyond Shared Geometry: Predictive Recovery, Exact Actions, and Registration Preference in Language-Model Representations
Year: 2026
Type: preprint
Abstract: Predictive maps between language-model representations need not identify the transformation structure those representations support. We study finite-group actions on visible input states with coordinates shared across contexts. A spectral budget retaining complete real irreducible components bounds the minimum prediction risk over exact orthogonal actions of a fixed real dimension. Under a common second-moment-whitened loss, its upper endpoint is at most twice its lower endpoint. An analytical example shows that the ordinary orbit-Gram spectrum does not determine this exact-action risk. On fixed banks from six frozen Llama and Qwen models, we separate independent orthogonality cost from bounds on the further cost of common group laws. These finite-bank class comparisons are distinct from held-out prediction. Training-constructed exact actions predict held-out representations less accurately than training-fitted independent least squares. Spectral registration preference depends on retained dimension and state-invariant components. At dimension 16, within-template centering with the original whitening changes the prescribed image's unique leads from 7/24 to 15/24 weekday (C₇) conditions and from 16/24 to 0/24 corner (D₄) conditions. Rewhitening raises the weekday count to 18/24 while corners remain at 0/24. On separately authored contexts, difference fitting lowers mean weekday error and raises mean corner error relative to coordinate fitting; wording, validation selection and, for corners, action weighting also differ from the earlier protocol. These comparisons distinguish predictive transport, fixed-bank exact-action compatibility and spectral preference from evidence of native computation.
Author: Yang, Pengcheng
Institution: Purdue University West Lafayette
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7233609048
Cites:
Cited by:
Related to:
