Year: 2026
Type: preprint
Abstract: We study decentralized bandit learning in discounted infinite-horizon, general-sum Markov games with continuous actions. Agents observe the public state, but only their own realized rewards, and use simultaneous perturbation stochastic approximation to update their actions. Under strong monotonicity of the negative running-reward gradients and sufficiently small time steps, we prove that the fully decentralized procedure converges almost surely to the unique stationary Markov perfect equilibrium. We obtain an $O(n^{-1/2})$ convergence rate and present a natural extension to episodic finite-horizon learning.
Source: arXiv (Cornell University)
Authors: Höfer, Felix, Soner, H. Mete
Institutions: University of Chicago, Princeton University
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
Funder: National Science Foundation
Award: Mean Field Optimal Control
OpenAlex ID: W7221800776