The Deadline-Carnot Theorem: A Finite-Time Thermodynamics of Compute-Optimal Training Under a Wall-Clock Constraint
Year: 2026
Type: preprint
Abstract: UNCERTIFIED — no hash-bound Viridis Comparator certificate covers this deposit's claims. Lean sources may compile but have not been independently certified. Results are conditional on the stated model assumptions. Historical description (verification assertions below are not current certification labels): Paper supported by machine-verified Lean 4. Compute-optimal scaling laws (Kaplan et al.; Hoffmann et al., "Chinchilla") prescribe a model-size/data-size split that minimizes loss for a fixed FLOP budget, implicitly assuming the training run may take arbitrarily long wall-clock time to realize that budget. Real training is instead bound by a hard deadline: a competitive release date, a funding window, a fixed rental period on a GPU fleet. We show that once training is required to finish within a wall-clock time tau, McCandlish et al.'s gradient-noise-scale theory of the steps/examples tradeoff becomes formally an endoreversible heat engine: batch size plays the role of a throttle, the number of optimizer steps plays the role of cycle time, and there is an absolute, un-crossable wall-clock floor tau_rm floor=S_min,t_1,N below which no finite batch size — however large — can complete training. The resulting data efficiency eta(tau)equiv D_min/D_rm used(tau) takes the exact Carnot functional form eta(tau)=1-tau_rm floor/tau, with the deadline tau playing the role of a hot reservoir and tau_rm floor the role of a cold one. Pricing wall-clock time against extra data tokens and minimizing total cost yields a closed-form golden-rule deadline tau^ast=tau_rm floor+sqrtc_D D_mintau_rm floor/c_tau — a genuine Curzon–Ahlborn-flavored square-root correction, derived rather than assumed, from the same generic near-a-critical-floor optimization that produces efficiency-at-maximum-power in physical heat engines. The theorem directly explains, in direction and mechanism, a 2026 empirical finding that wall-clock-constrained optimal model size grows differently than Chinchilla's FLOP-based prediction. All results are numerically verified (21/21); for an illustrative 18-billion-parameter model the golden-rule deadline sits at approx 30 days against an approx 18-day absolute floor, at a striking approx62% Deadline Tax in extra compute. We cast the construction as a new Intelligence Bound self-application, the Stoker, close with four testable predictions, and discuss implications for AI training infrastructure, conservation-model retraining under field-season deadlines, and the canon's own thermodynamic-economics program.
Authors: Hart, Justin D., Aristotle (Harmonic)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7171289897
Cites:
Cited by:
Related to: