Transient Low-Order Cues and Compositional Representation Onset: A Controlled Finite-Field Study
Year: 2026
Type: preprint
Abstract: What controls the accessibility of a compositional prediction rule when the target conditional and total predictable information are unchanged? We study this question in a finite-field, parent-exposed Transformer task by varying the early joint input sampling distribution, then returning all cue-bearing conditions to uniform full support at 5M exposures. The interventions preserve the target conditional, noise, parent marginals, and total pairwise predictable information, while changing lower-order predictive cues and joint distribution structure together. In eight prospectively specified design cells sharing only two distinct initialization seeds, the registered cue levels C_single = 0, 0.016, and 0.065092 yielded verified onset by 10M in 1/8, 6/8, and 8/8 trajectories, respectively. Verified onset requires normalized full-grid risk R < 0.9 and layer-6 linear-probe accuracy > 0.5 at two successive registered probe checkpoints. Mean full-grid normalized KL risks at 10M were 0.432743, 0.110766, and 0.005181. All 13 cue-bearing trajectories already threshold-positive at 5M remained positive at both 6M and 10M under uniform continuation. The registered Page-style label-permutation reference has an exact upper-tail fraction of 16/6⁸, but the frozen protocol does not establish the joint label exchangeability required for a calibrated significance p-value. We therefore report bounded computational observations, not a statistically confirmed population effect or identified causal mediator. This case motivates examining the joint sampling law and the time course of representation formation alongside information-budget summaries. The accompanying limited verification supplement provides the eight reported onset triples, a standalone reference-count verifier, and analytical checks of the finite-field controls. The original training trajectories and checkpoints remain in the local archive.
Author: Gao, Yongjing
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225386542