The Unassailable Case for PCIe and CXL in Single-Root Scale-Up Systems
Year: 2025
Type: article
Abstract: This white paper presents a definitive, technically rigorous argument to unequivocally prove the following statement: If your goal is to build a scale-up accelerator system under a single root complex, Ethernet, RoCE, or PCIe-over-Ethernet are not just inefficient—they are fundamentally wrong tools. These technologies violate the physics, semantics, and timing model that make a single-root system possible. The correct and only viable technologies for this architecture are PCIe Gen5/6 and CXL 3.0 (or NVLink/NVSwitch in GPU-centric designs). All other interconnects, including Ethernet, RoCE, SmartNICs, and DPUs, belong outside the server, connecting nodes together, not replacing the bus inside them. This paper will demonstrate, with scientific precision and hardware-level evidence, that Ethernet-based solutions cannot technically or physically substitute for PCIe/CXL in a single-root system. We will provide quantitative comparisons, signal flow diagrams, and latency/power measurements from verified public specifications and white papers to make this case unassailable. The frequent misunderstanding of this topic stems from the conflation of scale-up (intra-node) and scale-out (inter-node) architectures, a confusion this paper will decisively clear.
Author: Emilio Billi
Language: English
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7091554692
FWCI:
Cites:
Cited by:
Related to: