Year: 2026
Type: preprint
Abstract: In this work, we study the neural scaling laws of RydbergGPT, an autoregressive transformer model trained on qubit projective measurement data gathered from interacting Rydberg atom arrays. The quantum system is known to exhibit a finite-size remnant of a critical point as the laser detuning parameter is varied. We find that near the critical point the transformer loss as a function of training dataset size is well described by a power-law with a loss floor correction. However, away from criticality the quality of the power-law description is substantially reduced. We then compare the statistical structure of both Rydberg measurements and natural-language corpora using an entropy-normalised, finite sample corrected mutual information "two-point" function. We find that near-critical statistics of the two point functions are closest to those observed in natural-language, whilst other qubit configurations far from the critical point have two-point functions that decay more rapidly. This supports the hypothesis that multi-scale dependence contributes to stable neural scaling, and that scaling behaviour should be viewed as a property of the model-data pair.
Source: arXiv (Cornell University)
Topic: Quantum many-body systems
Subfield: Atomic and Molecular Physics, and Optics
Field: Physics and Astronomy
Domain: Physical Sciences
Open Access status: green
Funders: National Science Foundation, Innovation, Science and Economic Development Canada, Government of Canada, Ministero dello Sviluppo Economico, National Science and Technology Council, Institut Périmètre de physique théorique, Natural Sciences and Engineering Research Council of Canada, Science and Technology Facilities Council, Kavli Institute for Theoretical Physics, University of California, Santa Barbara
OpenAlex ID: W7213873301
Cites:
Cited by:
Related to: