Graphon Mean-Field Subsampling for Cooperative Heterogeneous Multi-Agent Reinforcement Learning
Year: 2026
Type: preprint
Abstract: Coordinating large populations of interacting agents is a central challenge in multi-agent reinforcement learning (MARL), where the size of the joint state-action space scales exponentially with the number of agents. Mean-field methods alleviate this burden by aggregating agent interactions, but these approaches assume homogeneous interactions. Recent graphon-based frameworks capture heterogeneity, but are computationally expensive as the number of agents grows. Therefore, we introduce $\texttt{GMFS}$, a $\textbf{G}$raphon $\textbf{M}$ean-$\textbf{F}$ield $\textbf{S}$ubsampling framework for scalable cooperative MARL with heterogeneous agent interactions. By subsampling $κ$ agents according to interaction strength, we approximate the graphon-weighted mean-field and learn a policy with sample complexity $\mathrm{poly}(κ)$ and optimality gap $O(1/\sqrtκ)$. We verify our theory with numerical simulations in robotic coordination, showing that $\texttt{GMFS}$ achieves near-optimal performance.
Source: arXiv (Cornell University)
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: National Science Foundation
Awards: CAREER: From Dynamic Algorithms to Fast Optimization and Back, Collaborative Research: CNS Core: Medium: Dynamic Data-driven Systems - Theory and Applications, Collaborative Research: CNS Core: Small: Optimizing Large-Scale Heterogeneous ML Platforms, Collaborative Research: CPS: Medium: Enabling DER Integration via Redesign of Information Flows
OpenAlex ID: W7130540429
Cites:
Cited by:
Related to: