Work

Percolation Markov Decision Processes

Year: 2026

Type: preprint

Abstract: We study Percolation Markov Decision Processes (PMDPs), in which the decision maker repeatedly moves a token through the d-dimensional integer lattice with deterministic transitions and random payoffs assigned to the edges. Payoffs are revealed before the beginning of the decision process and the decision maker aims to maximize the average accumulated payoff over a fixed horizon. We establish the existence of the uniform value (as the horizon tends to infinity) and 0-optimal strategies. In the particular case of Bernoulli payoffs, we establish continuity results for the uniform value. We also present several one-dimensional examples to illustrate that optimal strategies may be very complex. Then, we introduce a dimensional lifting property that allows PMDPs to be approximated by PMDPs of lower dimension and could be used to approximate critical probability thresholds in percolation theory.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2609.19905

Authors: García, Melissa González, Vigeral, Guillaume

Language: English

Topic: Stochastic processes and statistical mechanics

Subfield: Mathematical Physics

Field: Mathematics

Domain: Physical Sciences

Open Access status: green

Funders: European Commission, Agence Nationale de la Recherche

Award: Zoology of Algorithmic Methods for Dynamic Games

OpenAlex ID: W7213609502

Cites:

Cited by:

Related to: