The Crossover Episode: When Persistent Harness State Pays for Itself
Year: 2026
Type: preprint
Abstract: . Prior work argued that the extra cost of a stateful orchestration harness is an investment recovered at a crossover episode N*, but left N* unmeasured. Concurrent work on efficient agent harnesses reports large token savings from removing redundant reasoning callbacks, with development cost amortised within a single evaluation population. We isolate which mechanism earns the return. We pre-register an eight-episode, three-arm protocol on a deterministic 25-function benchmark with one fixed model (deepseek-flash, hidden reasoning disabled). All arms decompose work into one function per response; they differ only in harness state. F discards state after each episode; R persists verified decisions and executes them through native re-verification in place of a model callback (guarded decision reuse plus certified callback substitution); G adds a proposer, a disjoint acceptance gate and persisted prompt rules --- the stateful RSI team. Two per-response budgets (512 and 256 tokens) were fixed before the runs. Under both budgets the crossover exists and equals N* = 7. The return is almost entirely attributable to decision reuse, not to the RSI layer: at 512 tokens cumulative cost is 9,133 (F), 3,547 (R) and 7,347 (G) tokens, i.e. R saves 61.16% and G saves only 19.56% against F; at 256 tokens the figures are 9,046 / 3,734 / 7,228 and 58.72% / 20.10%. The entire F-to-G difference is the development cost of proposals and paired gate evaluations (3,859 and 3,603 tokens, paid in the first three episodes). Across both conditions the gate accepted zero of six candidate rules, so G's per-episode cost stays within 16% of R's throughout. The pre-registered prediction that gated persistence adds value over reuse alone is therefore not supported on this benchmark. We derive the exact identity N* = min{N : sum_{i<=N}(C_F,i - C_G,i) > C_dev}, show that the observed N* = 7 exceeds the analytical lower bound C_dev / mean(delta C) of about 5.5 because of cold start, and explain why the same mechanism pays off within 96 tasks in a visual-agent report while costing more than half the total budget here: N* is a ratio of development cost to per-task saving, not a property of a method. Artifact. The package contains the pre-registered experiment script, the benchmark and model client, and the two complete raw result files. Every number in the paper is read from those files.
Author: Wang, Jason
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225752217