Work

A case for server-scale photonic connectivity

Year: 2024

Type: conference-paper

Abstract: The commoditization of machine learning is fuelling the demand for compute required to both train large models and infer from them. At the same time, scaling the performance of individual microprocessors to satisfy the demand for compute has become increasingly difficult since the end of Moore's law and Dennard scaling. As a result, compute resources in modern servers are distributed across multiple accelerators on the server board. In this work, we make the case for using optics to interconnect accelerators within a server. A key benefit of on-board chip-to-chip optical connectivity is its ability to dynamically allocate bandwidth between accelerators, where necessary, rather than the common practice of statically dividing bandwidth among links within the topology of a multi-accelerator server, as seen in popular direct-connect architectures. This property prevents bandwidth under-utilization in state-of-the-art rack-scale multi-accelerator deployments. Moreover, server-scale optical connectivity can reduce the blast radius of individual accelerator failures in rack-scale ML deployments. Our early experiments with the prototype of a newly commercialized server-scale photonic interconnect show how the capability of the hardware can enable our vision.

Source: ACM Workshop on Hot Topics in Networks (HotNets)

DOI: 10.1145/3696348.3696856

Authors: Abhishek Vijaya Kumar, Arjun Devraj, Darius Bunandar, Rachee Singh

Institutions: Cornell University, Lightmatter (United States)

Language: English

Topic: Interconnection Networks and Systems

Subfield: Computer Networks and Communications

Field: Computer Science

Domain: Physical Sciences

SDG: Industry, innovation and infrastructure

Open Access status: hybrid

Funder: Universitas Brawijaya