Reconciling a model-validation engine against reference engines and printed answers: measured agreement, listed gaps and the size and power of the acceptance rule
Year: 2026
Type: preprint
Abstract: A validation function checks an engine against printed tables, second computations, closed forms or Monte Carlo reference engines; the result is worth what the reference shares with the engine and the rule's size and power. A protocol fixes a rule per class of reference and gives each pass an equivalence margin. On a bank's validation engine 313 of 438 printed numbers agree to the digit and 25 of 125 gaps have a tested explanation. Independent recomputations of tail probabilities agree to 5.7 x 10^-12 and flag a planted 10^-9 error that the engine's own call misses. Three of six hypotheses held; three are not met as stated: closed forms match QuantLib on 296 of 297 grid cases, a printed constant of Hull and White is 1.32 units from its digit, and the printed count differs. A single swap and a four-swap netting set agree with ORE within 1.9 % and 2.1 % of their EEPE; the 10-day, 20-day and initial-margin variants differ by up to 9.93, 7.09 and 11.74 combined standard errors, and the first fails on 8 of 8 new seeds. The acceptance rule against ORE rejects two runs of one model in 12.5 % [9.6, 16.1] of 400 repetitions and detects a 2 % volatility error in 75.0 % [70.5, 79.0]; pair-aware errors with a family-wise bound give 3.8 % [2.3, 6.1] and 76.5 % [72.1, 80.4]; a maximum-T bound on pair-aware errors, with covariance from pilot runs of the engine against itself, gives 4.8 % [3.1, 7.3] and 81.2 % [77.1, 84.8]. One synthetic book is covered. Repository and reproducible pipeline (MIT): https://github.com/fsantibanezleal/CAOS_Contraste .
Author: Santibáñez-Leal, Felipe
Language: English
Subfield: Finance
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7225621281