Work

RTT- or Bandwidth-Bound? Demystifying the KV Cache Transfer in Large Language Model Serving

Year: 2025

Type: conference-paper

DOI: 10.1145/3748273.3749196

Authors: Shengnan Yue, Mowei Wang, Yu Yan, Weiqiang Cheng, Zihan Jiang, Zhenhui Zhang

Institutions: China Mobile Research Institute, Huawei Technologies (China), Nanjing University

Language: English

Topic: Parallel Computing and Optimization Techniques

Subfield: Hardware and Architecture

Field: Computer Science

Domain: Physical Sciences

Open Access status: closed

Funder: National Key Research and Development Program of China

Award: G6804312440

OpenAlex ID: W4413921941

FWCI: 0.928

Cites: 6

Cited by: 1

Related to: 10