When Free Executors Cost More: The Free-Executor Paradox in Iterative LLM Code-Repair Loops
Year: 2026
Type: preprint
Abstract: We compare four LLM configurations on three Python code-repair tasks (breakage recovery, refactor, feature-add) using a single deterministic correctness judge (mypy + ruff + pytest exit codes). The configurations are two single-model 'solo' runs of Anthropic Opus 4.7 and Haiku 4.5, and two 'orchestrator + executor' structures in which Opus orchestrates either local Qwen 3.5-9B (zero marginal cost) or Anthropic Haiku 4.5. Across 40 trials with n=3 successful runs per cell, the cost/wall-time/iterations Pareto frontier varies across the three task types. On the smallest task, Opus solo dominates every cloud-only axis at $0.17/trial and 6 iterations median; only Haiku solo undercuts it on cost. On the longest task, Haiku solo wins on dollar cost (5.5x cheaper than the cheapest cloud arm), at a 25% failure rate within our per-arm iteration cap. The canonical 'strong orchestrator + cheap executor' structure (Opus + Qwen) is the most expensive cloud arm on every task, because the orchestrator's prompt-cached re-reads of the executor's returned summaries dominate any savings from delegating execution. The paper aims to make the underlying tradeoff cheap to measure on other codebases, not to crown a single winner.
Author: Imoto, Ken
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7166294375
Cites:
Cited by:
Related to: