Language as a Compression Artifact of Physical Reality: Category-Theoretic Nuisance Fibrations, the Platonic Representation Theorem, Procrustes Spectral Decay, and Cross-Scale Activation Patching
Year: 2026
Type: preprint
Abstract: Overview This 21-page research manuscript, theoretical synthesis, and Python benchmarking suite by Jeremiah Swann (Lead Researcher) investigates how neural language and vision models trained via gradient descent converge toward a shared latent geometry—and pinpoints the exact mathematical and layerwise boundaries where this structural isomorphism breaks down in smaller-parameter models. Grounded in the premise that human language evolved as an Information Bottleneck (Tishby et al., 2000) compression algorithm of physical reality, this work synthesizes the Platonic Representation Hypothesis (Huh et al., 2024), the Platonic Representation Theorem for JEPAs (Balestriero & LeCun, 2026), Information Bottleneck semantic compression dynamics (Shani et al., 2025), Relative Representations (Moschella et al., 2023), and unsupervised cycle-consistent embedding translation via vec2vec (Jha et al., 2025). Core Theoretical & Empirical Contributions Category-Theoretic Nuisance Fibrations ($\mathbf{1}^\perp = \mathcal{H}_S \oplus \mathcal{H}_N$): Models observation modalities inside a Markov category $\mathbf{Stoch}(\mathcal{Z}, p)$ equipped with canonical state-projection fibrations $s_m : \mathcal{X}_m \twoheadrightarrow \mathcal{Z}$ that separate the horizontal world-state subspace $\mathcal{H}_S$ from the vertical modality-nuisance fiber $\mathcal{H}_N$ (author wording/syntax in text; viewpoint/lighting in vision). Universal Coequalizer Factorization vs. Next-Token Bad-View Leakage: Integrates the proof by Balestriero & LeCun (2026) that good-view JEPAs (LeJEPA, Balestriero & LeCun, 2025) factorize uniquely through $s_m$ up to an orthogonal transformation $\mathbf{Q}_m \in O(k)$, whereas autoregressive next-token prediction acts as a bad-view JEPA with $\mathrm{GL}(k)$ gauge symmetry that leaks $r \ge 1$ principal directions into intra-document nuisance whenever $\beta_1 > \alpha_k$, imposing an irreducible Procrustes alignment floor of $2k(1 - \min_m \rho(f_m)) \ge 2r$ and requiring $42\times$ more training samples than good views (Ziyin & Chuang, 2025). Scale Splitting Index ($\mathsf{SSI}$) & Superposition Disentanglement: Extends cross-scale Sparse Autoencoder (SAE) universality benchmarks across Pythia (Biderman et al., 2023) and Gemma-2 (Riviere et al., 2024) established by Lan et al. (2025) and Son et al. (2025), defining the cumulative Scale Splitting Index ($\mathsf{SSI}_{\text{cum}}$) to track surjective ($\text{Many}_F\text{-to-}1_S$) feature splitting across normalized network depth. Orthogonal Procrustes Spectral Decay & Random Control Pipeline ($\Delta_{\text{Procrustes}}$): Implements closed-form Orthogonal Procrustes alignment (Procrustes Bounds, 2025), SVCCA (Raghu et al., 2017), Linear CKA (Kornblith et al., 2019), and mutual $k$-NN $\mathsf{CKNNA}$ (Huh et al., 2024) paired against a randomly initialized Pythia-70M control group ($\mathcal{M}_{\text{null}}$), demonstrating a peak learned optimization gain of $\Delta_{\text{Procrustes}} = +1.470$ ($p < 0.005$) in middle conceptual layers ($\tilde{\ell} = 0.50$). Cross-Scale Causal Activation Patching & Compression Breakdown Boundary ($\ell^*$): Grounds causal functional isomorphism in localized mid-layer MLP associative memory editing (ROME, Meng et al., 2022a; MEMIT, Meng et al., 2022b), projecting frontier residual updates into smaller experimental cohorts (Pythia, Gemma-2, and Qwen2, Yang et al., 2024) to isolate the exact layerwise threshold ($\ell^* = \text{L5}, \tilde{\ell}^* = 0.83$) where nuisance saturation triggers an $\mathsf{SSI}$ spike ($1.75 \to 2.97$) and collapses causal patching efficacy ($\eta_{\text{patch}} : 0.890 \to 0.000$). Included Files in Deposit shared_latent_space_manuscript_swann.pdf / .tex — Full 21-page illustrated manuscript with Category-Theoretic Nuisance Fibrations, the Platonic Representation Theorem, formal hypotheses, and empirical telemetry. shared_latent_space_benchmark_harness.py — Self-contained Python benchmarking harness with --self-test verification and HuggingFace/PyTorch hooks. fig1_procrustes_random_control_delta.png, figure_2_singular_value_decay_spectra.png, figure_3_ssi_activation_patching_breakdown.png — 300-DPI publication figures.
Author: Swann, Jeremiah
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225416154