Replication package: The Price of Proof — Cost per Verifier-Accepted Function in LLM Generation of Formally Verified Code
Year: 2026
Type: preprint
Abstract: Replication package for "The Price of Proof: Cost per Verifier-Accepted Function in LLM Generation of Formally Verified Code". Benchmarks for large language model generation of formally verified code report how often a verifier accepts the result, but not what the acceptance cost. This package contains the complete measurements behind a study that prices it, on the vericoding benchmark (Dafny, Verus, Lean 4) across a seeded, difficulty-stratified 30-task sample, seven hosted API models and three prompting regimes. Contents: the complete per-call run ledger (one record per model call, with billed input, output and reasoning tokens, cost, verifier outcome and error class); per-unit, per-cell and per-model derived tables; the repair-depth economics and failure taxonomy; the task-clustered stratified percentile bootstrap used for every interval in the paper (B = 10,000, seed 20260810); the model registry and list prices at which every dollar was computed; the three figures; and the machine-generated LaTeX macro files the manuscript reads its numbers from. Every number in the manuscript reaches the page through one of those macro files, and both are regenerated from the data by the included scripts, which self-check by recomputing each quantity along two independent routes. Code is MIT licensed; data is CC BY 4.0. See LICENSE, LICENSE-DATA and LICENSES-AUDIT.md inside the archive. The underlying task specifications come from the vericoding benchmark under its own licence.
Source: arXiv (Cornell University)
Author: Patodiya, Aditi
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7084166387
Cites:
Cited by:
Related to: