Occupation-Weighted Performance Bounds for Rollout Policies in Stochastic Shortest Path Problems
Year: 2026
Type: preprint
Abstract: We study rollout policies constructed from an approximate value function for stochastic shortest path problems with an absorbing terminal state. Our main bound weights how much the selected action exceeds the optimal one-step value by expected state visits before termination; the uniform hitting-time estimate follows as a corollary. The bound separates value-approximation and numerical-optimization errors from the error caused by replacing an expectation with a nominal disturbance, and uses approximation errors only at states that can occur after one transition from states visited by the implemented policy. A positive lower bound on nonterminal stage costs also gives conditions under which the generated policy reaches the terminal state almost surely with finite expected total cost. In the minimum expected hitting-time problem, a uniform value-approximation error smaller than one half guarantees almost-sure termination, finite expected cost, and a multiplicative hitting-time bound, and the value one half cannot be increased. A deterministic construction shows that the factor two in the uniform rollout bound is asymptotically sharp. Certainty-equivalent rollout is covered by the same analysis.
Source: arXiv (Cornell University)
Authors: Hansson, Anders, Wahlberg, Bo
Institutions: Linköping University, KTH Royal Institute of Technology
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7162332191