Work

Strategy complexity of limsup and liminf threshold objectives in countable MDPs, with applications to optimal expected payoffs

Year: 2024

Type: article

Abstract: We study Markov decision processes with a countably infinite number of states. The $$\limsup $$ lim sup (resp. $$\liminf $$ lim inf ) threshold objective is to maximize the probability that the $$\limsup $$ lim sup (resp. $$\liminf $$ lim inf ) of the infinite sequence of directly seen rewards is non-negative. We establish the complete picture of the strategy complexity of these objectives, i.e., the upper and lower bounds on the memory required by $$\varepsilon $$ ε -optimal (resp. optimal) strategies. We then apply these results to solve two open problems from (Sudderth in Decis Econ Finan 43:43–54, 2020) about the strategy complexity of optimal strategies for the expected $$\limsup $$ lim sup (resp. $$\liminf $$ lim inf ) payoff.

Source: Decisions in Economics and Finance

DOI: 10.1007/s10203-024-00485-7

Authors: Richard Mayr, Eric Munday

Institution: University of Edinburgh

Language: English

Topic: Formal Methods in Verification

Subfield: Computational Theory and Mathematics

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $3,090

Funder: Royal Society