Safe and Programmable OS-Level GPU Resource Management with eBPF
Year: 2026
Type: conference-paper
Abstract: GPU resource management policies govern how a system places memory and schedules GPU tasks. They play an important role in determining application performance, and the ideal policy for a given deployment often depends on low-level details such as the application, workloads, and hardware. AI agents would make it possible to automatically optimize GPU resource management policies, but require three properties of a system: safety, to ensure that a broken policy cannot cause a system crash; dynamism, to enable policy changes without restarts; and full-stack, to ensure that the policy can observe device and application states and control low-level hardware. Unfortunately, existing techniques for GPU resource management fail to meet these needs. We introduce gpu_ext, a new system that provides an OS interface for GPU resource management that is safe, dynamic, and full-stack by turning to eBPF: gpu_ext policies include eBPF callback functions that the system invokes on a specified event (e.g., when code reaches a function). The system includes a novel execution model that separates the effect of a callback from its triggering event. Namely, gpu_ext supports policies that asynchronously transition resources between states, potentially invoking multiple transitions at a single point-in-time. We implemented gpu_ext on the Linux NVIDIA GPU driver and find that AI-generated policies improve throughput by 1.76× over application-tuned baselines and reduce tail latency by up to 2×, without application modifications. Reimplementations of seven prior policies with gpu_ext achieve performance close to their native implementations. Moreover, we find that there were zero OS kernel panics through the optimization process, highlighting gpu_ext's safety.
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7225256299