Auditable Release Assurance for Quantized LLM Deployment Configurations: Anytime-Valid Non-Inferiority and Executable Certificates
Year: 2026
Type: preprint
Abstract: Quantization changes not only model size but the deployed software configuration , yet release decisions are often based on a single benchmark gap. We present an auditable software-quality gate for deciding whether a quantized large language model (LLM) may replace a reference configuration under a declared non-inferiority margin ε and false-certification budget δ. The method uses paired item-level outcomes and supports two distinct targets: future requests sampled from a task distribution and a committed finite audit frame inspected without replacement. Fixed-sample and anytime-valid procedures control false certification under their stated assumptions, while precommitment, candidate-selection rules, and a lifetime risk ledger address optional stopping and repeated releases. Protocol 3.1 binds the decision to a versioned deployment profile and to the supplied task, evaluator, frame, selection, and ordered-record files; a verifier that can be run by an independent third party replays the reference implemen-tation’s bindings, selected candidate, inspection order, and sequential-evidence checks. We evaluate the procedure on twelve measured model–task–quantization populations spanning NVIDIA Jetson Orin Nano and Mac deployment configurations. For the proposed procedures, the maximum observed false-certification rate across the simulated null-region cells was 3.8% for a 5% budget. Representative 500-item audit frames reached retrospective stopping points between 167 1 and 423 items. Cross-configuration parsed-prediction disagreement for the same quantized artifact reached 11.8%, supporting configuration-specific rather than file-specific release evidence. The current certificate example is a retrospective replay; prospective assurance additionally requires external anchoring.
Source: Research Square
Institution: Kyungpook National University
Language: English
Subfield: Safety, Risk, Reliability and Quality
Field: Engineering
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225824404
Cites:
Cited by:
Related to: