Did You Actually Compute That? Numerical Admissibility for AI Systems
Year: 2026
Type: preprint
Abstract: We present AP-1, an open protocol and reference instrument for determining whether an AI-reported number was produced by the expected computation, thus establishing auditable provenance. This is separate from validation, as a correct number does not prove a correct calculation occurred. An AI system's offered result may be derived from authoritative inputs fed through an approved algorithm, copied from current or past context, generated spontaneously without support, or computed correctly from incorrect inputs. Post-hoc model explanations do not reliably separate these cases as required by regulatory audits and high-consequence applications. AP-1 addresses this challenge through the concept of numerical admissibility: a result is admissible when evidence establishes an intended computation produced a reported value via admissible operands. AP-1 measures this property at the operand boundary, where values enter the computation. AP-1 does not judge pass/fail directly, instead gathering robust evidence to enable that determination. The method was deployed on two AI systems across 2,000 controlled executions, repeated once after the defects were corrected. In one perturbation experiment, removing the instruction to use an available calculator was associated with one AI system skipping the calculator in 92 of 500 executions, against 0 of 500 with the instruction. The second system never skipped it under either condition. The evaluation also exposed defects in our own tooling, and some results were withdrawn. Overall accuracy testing alone cannot establish how a reported number was produced, and thus cannot provide computational assurance. AP-1 addresses that need.
Authors: Rupp, Marcus, Mandrake, Lukas, Lewis, Steven
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225462695