From Fixed Points to Scaling Laws: A Parameter-Free Prediction of Grokking Phase Transitions in Large Language Models Using the Functional Renormalization Group
Year: 2026
Type: preprint
Abstract: The empirical success of Large Language Models (LLMs) is currently governed by phenomenological scaling laws and heuristic observations of phase transitions, such as "Grokking". However, a rigorous, first-principles mathematical framework guaranteeing the stability and universality of these dynamics remains absent. In this work, we bridge this gap by formulating the training dynamics of overparameterized neural networks as a statistical field theory on the Fisher Information Metric (FIM) manifold. We derive the exact Wetterich flow equation for the effective average loss, demonstrating that macroscopic scaling laws emerge parameter-free as critical exponents of a non-trivial infrared (IR) fixed point. Furthermore, we prove that the Grokking transition is not an optimization artifact, but a strict Non-Equilibrium Steady State (NESS) Lyapunov attractor, driven by the discontinuous collapse of the Hessian's spectral entropy. By establishing the completeness of the O(∂²) truncation, we quantify finite-size corrections to scaling laws. Finally, we provide three concrete, falsifiable predictions regarding the Hessian spectral gap, entropy collapse, and RG data collapse, offering a mathematically closed foundation for AI safety and core mechanism research.
Author: Hasić, Alis
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225423630