Work

Distributed Training of Large Graph Neural Networks With Variable Communication Rates

Year: 2025

Type: article

Abstract: Training Graph Neural Networks (GNNs) on large graphs presents unique challenges due to the large memory and computing requirements. Distributed GNN training, where the graph is partitioned across multiple machines, is a common approach to training GNNs on large graphs. However, as the graph cannot generally be decomposed into small non-interacting components, data communication between the training machines quickly limits training speeds. Compressing the communicated node activations by a fixed amount improves the training speeds, but lowers the accuracy of the trained GNN. In this paper, we introduce a variable compression scheme for reducing the communication volume in distributed GNN training without compromising the accuracy of the learned model.

Source: IEEE Transactions on Signal and Information Processing over Networks

DOI: 10.1109/tsipn.2025.3557584

Authors: Juan Cerviño, Md Asadullah Turja, Hesham Mostafa, Nageen Himayat, Alejandro Ribeiro

Institutions: Massachusetts Institute of Technology, University of North Carolina at Chapel Hill, Intel (United States), University of Pennsylvania

Language: English

Topic: Advanced Graph Neural Networks

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W4409122044

FWCI: 0.408

Cites: 65

Cited by: 1

Related to: 10