Communication-Efficient Distributed Computing Through Combinatorial Multi-Access Models
Year: 2025
Type: preprint
Abstract: This paper explores the multi-access distributed computing (MADC) model, a novel distributed computing framework where mapper and reducer nodes are distinct entities. Unlike traditional MapReduce frameworks, MADC leverages coding-theoretic techniques to minimize communication overhead without necessitating file replication across mapper nodes. We introduce a new approach utilizing combinatorial designs, specifically t-designs, to construct efficient coding schemes that achieve a computation load of 1. By establishing a connection between t-designs and MapReduce Arrays, we characterize the achievable communication loads and demonstrate the flexibility of our method in selecting the number of reducer nodes. The proposed scheme significantly reduces the number of reducer nodes relative to existing combinatorial topology schemes, at the expense of increased communication cost.
Source: arXiv (Cornell University)
Authors: Sasi, Shanuja, Günlü, Onur
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6966957994
Cites:
Cited by:
Related to: