Year: 2025
Type: preprint
Abstract: The rapid growth of large language models (LLMs) has driven the need for high-performance, scalable GPU hardware capable of efficiently serving models with hundreds of billions of parameters. While NVIDIA GPUs have traditionally dominated LLM deployments due to their mature CUDA software stack and state-of the-art accelerators, AMD's latest MI300X GPUs offer a compelling alternative, featuring high HBM capacity, matrix cores, and their proprietary interconnect. In this paper, we present a comprehensive evaluation of the AMD MI300X GPUs across key performance domains critical to LLM inference including compute throughput, memory bandwidth, and interconnect communication.
Source: arXiv (Cornell University)
Authors: Ambati, Chandrish, Diep, Trung
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4415927754
Cites:
Cited by:
Related to: