Time and Money in LLM-Generated Code: Benchmark Protocol v1 (Pre-registration)
Year: 2026
Type: article
Abstract: Frozen protocol of the benchmark "Time and Money in LLM-Generated Code", deposited before the main experiments. No benchmark task was sent to any model before this deposit; the pilot used four separate pilot tasks.Contents: 20 task specifications with decided-case tables, prompt templates (C0, C1, C2, T), two reference implementations (Python, TypeScript), test sets (examples, boundary, property, mutated, generic PBT), mutants, the model list with pinned providers and sampling settings, the analysis plan, sandbox runner code, and deviations from the draft (DEVIATIONS.md). MANIFEST.json lists the SHA-256 of every file and the git commit.
Author: Pidvolotska, Violetta
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225232385
FWCI:
Cites:
Cited by:
Related to: