Work

Heterogeneous Multi-agent Multi-armed Bandit on Stochastic Block Models

Year: 2025

Type: article

Abstract: We study a novel heterogeneous multi-agent multi-armed bandit problem with a cluster structure induced by stochastic block models (SBMs), influencing not only graph topology, but also reward heterogeneity. Specifically, agents are distributed on random graphs based on SBMs, a generalized Erdos-Renyi model with heterogeneous edge probabilities: agents are grouped into clusters (known or unknown); edge probabilities for agents within the same cluster differ from those across clusters. The same cluster structure in SBMs also determines our heterogeneous rewards. Rewards distributions of the same arm vary across agents in different clusters but remain consistent within a cluster, unifying homogeneous and heterogeneous settings and varying degree of heterogeneity. Rewards are independent samples from these distributions. The objective is to minimize system-wide regret across all agents. To address this, we propose a novel algorithm applicable to both known and unknown cluster settings. The algorithm combines an averaging-based consensus approach with a newly introduced information aggregation and weighting technique, resulting in a UCB-type strategy. It accounts for graph randomness, leverages both intra-cluster (homogeneous) and inter-cluster (heterogeneous) information from rewards and graphs, and incorporates cluster detection for unknown cluster settings. We derive optimal instance-dependent regret upper bounds of order log T under sub-Gaussian rewards. Importantly, our regret bounds capture the degree of heterogeneity in the system (an additional layer of complexity), exhibit smaller constants, scale better for large systems, and impose significantly relaxed assumptions on edge probabilities.

Source: Proceedings of the ACM on Measurement and Analysis of Computing Systems

DOI: 10.1145/3771570

Authors: Mengfan Xu, Liren Shan, Fatemeh Ghaffari, Xuchuang Wang, Xutong Liu, Mohammad Hajiesmaili

Institutions: University of Massachusetts Amherst, Toyota Technological Institute at Chicago, University of Washington Tacoma

Language: English

Topic: Advanced Bandit Algorithms Research

Subfield: Management Science and Operations Research

Field: Decision Sciences

Domain: Social Sciences

Open Access status: hybrid

Funder: National Science Foundation

Awards: CAREER: A Robust and Data-driven Design for Carbon-intelligent Distributed Systems, Collaborative Research: NeTS: Small: CoLeNe: Cooperative Learning in Heterogeneous Edge Networks, Collaborative Research: SaTC: CORE: Small: Corruption-Robust Online Optimization: from Theory to Applications, National Science Foundation Expeditions in Computing for Computational Decarbonization of Societal Infrastructures at Mesoscales