Training Trajectory Structure: Regime Detection, Cross-Seed Convergence, and Speculative Weight Prediction
Year: 2026
Type: article
Abstract: Training trajectories alternate between stable basins where gradients are predictable and chaotic transitions where they are not. We show these regimes are detectable via activation fingerprints, synchronized across independent seeds, and exploitable for speculative weight prediction. Consolidates and supersedes the Leap+Verify (DOI: 10.5281/zenodo.18828410) and Ensemble Collapse (DOI: 10.5281/zenodo.18828412) papers into a unified treatment. Linear prediction achieves 60-90% strict acceptance at 7B scale in stable regimes. Momentum-based prediction fails catastrophically at all scales (100-10,000x loss inflation). Phase boundaries synchronize across 5 independent seeds to within +/-50 steps. Final validation loss CV is 0.41% (124M), 2.43% (1.5B), 1.53% (7B).
Author: McEntire, Jeremy
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7150062083