Get started
Work

Execution Information and Estimation Timing for Coding Agent ETA: An Empirical Study and a Delivery-Aware Framework

Year: 2026
Type: preprint
Abstract: Coding agents produce execution histories that can inform estimates of their remaining runtime. Designing a useful ETA service requires understanding which information improves forecasts, when it becomes useful, and how prediction latency affects delivery. We study these questions through same-anchor information comparisons, a four-model timing study, and frozen-trace replay, and implement a framework linking observable execution prefixes to delivered countdowns. Our retrospective synthesis evaluates raw error, signed bias, task discrimination, and display coverage. Richer execution packages reduce raw error in several settings: at 60 seconds, mean absolute error falls from 396.0 to 189.7 seconds on eleven separate confirmation tasks. The benefit varies with estimation time and execution environment. Canonical replay shows that later error reductions can primarily reflect less overestimation, without an established improvement in task discrimination. Across four models, absolute error decreases at later stages, while relative accuracy is not uniformly monotonic. Delivery analysis further shows why improved point forecasts need an evaluated update policy to benefit a displayed countdown. We contribute a joint empirical characterization of information and timing, a measurement approach separating error reduction from bias and discrimination, and an auditable ETA framework. The evidence concerns the studied information packages and trajectories; originally frozen calibrated primary tests remain inconclusive. Preprint; not peer reviewed. Version 1.7. Code and companion numerical and methods materials: https://github.com/kinchinglam66-alt/codex-progress-hud/releases/tag/research-release-2026-10-24
DOI: 10.5281/zenodo.23613203
Language: English
Open Access status: green
OpenAlex ID: W7223477321
Cites:
Cited by:
Related to: