Empirical Results from 31 Rounds of Multi-Model Structural Honesty Audit
Year: 2026
Type: article
Abstract: Empirical results from 31 audit rounds on the XZ Protocol, a construct-validation preregistration for structural-decision replay of the XZ Utils supply-chain compromise (CVE-2024-3094). The audit program produced 199 seat reports from five core vendor families (Anthropic, OpenAI, Google, Moonshot, Zhipu), yielding 832 structured findings (136 Class A byte-false claims). Primary result: 25 consecutive rounds (R7--R31) with Class A present, with 102 extracted Class A findings across 19 rounds and description-level or report-metadata evidence for 5 additional rounds (1 round ASSERTED from the companion Heartbeat paper). Secondary result: defect migration trajectory with recurrence across code/CI (41 findings), cross-layer consistency (23), methodology/dispatch (11), provenance/amendment (6), and unclassified (21) locus layers. The deposit includes the paper, per-round recurrence witness table, per-finding migration classification, the classification script, and the structured corpus data. Four audit rounds across three non-producer model families; carrying seat returned CONDITIONAL PASS at R4. This paper makes no effectiveness claim about any component of the Structural Honesty program.
Author: Arfeen, Bilal Syed
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7203918979
FWCI:
Cites:
Cited by:
Related to: