Unichain and Aperiodicity Are Sufficient for Asymptotic Optimality of Average-Reward Restless Bandits
Year: 2025
Type: article
Abstract: We consider the infinite-horizon, average-reward restless bandit problem in discrete time. We propose a new class of policies that are designed to drive a progressively larger subset of arms toward the optimal distribution. We show that our policies are asymptotically optimal with an [Formula: see text] optimality gap for an N-armed problem, assuming only a unichain and aperiodicity assumption. Our approach departs from most existing work that focuses on index or priority policies, which rely on the Global Attractor Property to guarantee convergence to the optimum, or a recently developed simulation-based policy, which requires a Synchronization Assumption. Funding: Y. Hong and W. Wang are supported in part by the U.S. National Science Foundation (NSF) [Grants ECCS-2145713, CCF-2403194, CCF-2428569, and ECCS-2432545]. Y. Chen is supported in part by the NSF [Grant CCF-2233152]. Q. Xie is supported in part by the NSF [Grants CNS-1955997, ECCS-2339794, and ECCS-2432546]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/moor.2024.0678 .
Authors: Yige Hong, Qiaomin Xie, Yudong Chen, Weina Wang
Institutions: Carnegie Mellon University, University of Wisconsin–Madison
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
Awards: CAREER: Achieving Ultra-Low Latency under Heterogeneity and Uncertainty in Edge Computing, Collaborative Research: New Algorithms and Theory for Weakly Coupled Markov Decision Processes, CNS Core: Medium: Resource Constrained Reinforcement Learning for Computing Systems, CAREER: Embracing Local Minima and Nonsmoothness in Nonconvex Statistical Estimation: From Structures to Algorithms, Collaborative Research: CIF: Medium: Towards optimal scheduling for parallelizable machine learning training workloads, CAREER: Stochasticity and Resilience in Reinforcement Learning: From Single to Multiple Agents
OpenAlex ID: W4417246550