Theorems, Not Tests: Kernel-Checked Guarantees for a Pretrained Language Model and a Neural PDE Solver
Year: 2026
Type: preprint
Abstract: We prove the first theorems about the outputs of a released pretrained language model, and the first error bound for a trained neural PDE solver in which a proof kernel checks every numerical step, both in Lean 4. (1) For the released Pythia-14M checkpoint, all 14,067,712 parameters at their stored float16 values and the complete six-layer forward pass with exact GELU, LayerNorm, rotary attention and softmax, we prove that on a 32-token prompt the prediction “ trained” beats each of the other 50,303 vocabulary tokens by a logit margin of at least 2, and that an explicit perturbation of one input embedding, of ℓ∞ norm 37/16384 (2.5% of that embedding’s own norm), makes “ dataset” beat it by at least 1/20: a formally proved adversarial example for a pretrained language model. (2) For a trained physics-informed neural network solving a two-dimensional Poisson problem, we prove that the error is below 1/30000 at every point of the closed unit square, against every solution of the boundary-value problem, and at least 1/80000 at a named point, so the certificate is within a proved factor 2.52 of the truth. The kernel’s enclosures stay within 2·10⁻¹⁴ of the exact values through all six transformer layers. Both theorems rest only on propext, Classical.choice and Quot.sound, with no native evaluation and no numerical hypothesis; they take 19.5 and 42.7 CPU-hours to check and have been replayed from source. Code: https://github.com/leochlon/tiny-bayes
Authors: Karim, Ahmed, Chlon, Leon
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7236277937
