Year: 2023
Type: preprint
Abstract: This paper studies a cooperative multi-agent multi-armed stochastic bandit problem where agents operate asynchronously -- agent pull times and rates are unknown, irregular, and heterogeneous -- and face the same instance of a K-armed bandit problem. Agents can share reward information to speed up the learning process at additional communication costs. We propose ODC, an on-demand communication protocol that tailors the communication of each pair of agents based on their empirical pull times. ODC is efficient when the pull times of agents are highly heterogeneous, and its communication complexity depends on the empirical pull times of agents. ODC is a generic protocol that can be integrated into most cooperative bandit algorithms without degrading their performance. We then incorporate ODC into the natural extensions of UCB and AAE algorithms and propose two communication-efficient cooperative algorithms. Our analysis shows that both algorithms are near-optimal in regret.
Source: arXiv (Cornell University)
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
Awards: G1034401897, G1198325749, Collaborative Research: CNS Core: Small: Dynamic Pricing and Procurement for Distributed Networked Platforms, CAREER: A Robust and Data-driven Design for Carbon-intelligent Distributed Systems, G5259331294, G6671297155, Collaborative Research: CNS Core: Medium: Dynamic Data-driven Systems - Theory and Applications, Collaborative Research: CPS Medium: Enabling DER Integration via Redesign of Information Flows
OpenAlex ID: W4321162378