Work

Measuring GPU utilization one level deeper

Year: 2025

Type: preprint

Abstract: GPU hardware is vastly underutilized. Even resource-intensive AI applications have diverse resource profiles that often leave parts of GPUs idle. While colocating applications can improve utilization, current spatial sharing systems lack performance guarantees. Providing predictable performance guarantees requires a deep understanding of how applications contend for shared GPU resources such as block schedulers, compute units, L1/L2 caches, and memory bandwidth. We propose a methodology to profile resource interference of GPU kernels across these dimensions and discuss how to build GPU schedulers that provide strict performance guarantees while colocating applications to minimize cost.

Source: Repository for Publications and Research Data (ETH Zurich)

DOI: 10.3929/ethz-c-000793181

Authors: Elvinger, Paul, Strati, Foteini, Enright Jerger, Natalie, Klimovic, Ana

Language: English

Topic: Parallel Computing and Optimization Techniques

Subfield: Hardware and Architecture

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W7124766782

Cites:

Cited by:

Related to: