MAS-CP: Multi-Agent Communication Prediction for Distributed Deep Neural Network Training
Year: 2026
Type: conference-paper
Abstract: Predicting all-reduce communication latency in distributed deep neural network (DNN) training is challenging because latency depends on both shared network state and per-worker program behavior. Existing approaches rely on monolithic predictors with fixed architectures, making them brittle across models and cluster sizes. We propose MAS-CP, a multi-agent framework with Locator, Analyzer, Predictor, and Verifier agents plus an orchestrator and RAG memory. On realworld BERT/GPT-2 traces and simulated multi-worker clusters with worker counts $W=2,4,8,16$, MAS-CP reduces MAE by $\mathbf{3. 6 \% - 6. 1 \%}$ over the best fixed-configuration baselines, with a closed-loop verification mechanism that automatically triggers re-prediction when errors exceed a dynamic threshold.
Authors: Zijie Huang, Zixuan Chen, Kai Shi
Institutions: China Telecom (China), China Telecom, Alibaba Group (China)
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7204268483