Energy-Efficient GPU Resource Management for Generative AI on HPC Systems: A Roadmap Towards Cross-Layer Co-Design Approaches
Year: 2026
Type: preprint
Abstract: Generative AI (GenAI) workloads are emerging as a major source of resource demand in high-performance computing (HPC) systems, creating new challenges for energy-efficient GPU management because of their extreme scale, memory and communication intensity, and highly dynamic training and inference behavior. Existing approaches remain fragmented across individual system layers, including hardware power control, runtime adaptation, and cluster scheduling, limiting their ability to deliver system-wide energy efficiency for large-scale GenAI deployment. This paper presents a structured review of 46 representative studies on energy-aware GPU resource management for GenAI on HPC platforms. The literature is synthesized through a cross-layer taxonomy spanning system layer, workload phase, decision type, evaluation scale, and energy-efficiency technique. The analysis shows that prior work has advanced telemetry-driven measurement, energy-performance prediction, scheduling and allocation, runtime power control, and architectural specialization, with several studies reporting substantial energy savings under specific workload and system conditions. At the same time, persistent gaps remain, including limited cross-layer coordination, weak support for token-level and phase-aware workload dynamics, and insufficient integration of carbon-aware objectives into resource management. To address these limitations, the paper proposes GenAI-XEnergy, a conceptual framework that unifies telemetry, prediction, orchestration, runtime control, and heterogeneous accelerator support within a closed-loop, cross-layer architecture. The review contributes a focused taxonomy, a comparative synthesis of existing techniques, an explicit analysis of open challenges, and a practical foundation for future research on sustainable, energy-proportional GenAI execution in HPC environments.
Source: Research Square
Authors: Atha Shamas, M. Usman Ashraf
Institution: Government College Women University Sialkot
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7235367614
