Year: 2026
Type: article
Abstract: Shared GPU clusters often execute multiple distributed training jobs concurrently under fluctuating contention. We reinterpret this setting as a two-scale control problem, where the micro scale captures intra-job learning dynamics and the macro scale captures inter-job resource arbitration. We propose an entropy-guided hierarchical framework that links these two scales through a unified uncertainty signal computed from training logits. Unlike existing uncertainty-aware methods that typically use uncertainty for only a single level of decision making, our approach uses the same entropy-based signal to jointly support both intra-job adaptation and inter-job scheduling within a hierarchical control loop. At the micro level, each worker estimates predictive uncertainty via normalized entropy and converts it into stable weights that drive epoch-level controls for uncertainty-aware data sharding, fixed-budget batch-size reallocation, and learning-rate modulation, while remaining compatible with standard synchronous data-parallel training. At the macro level, the same signal is aggregated into a job utility score that guides admission, ordering, and GPU quota assignment under contention. In large-scale workload-driven simulation, our method reduces average job completion time (JCT) by 23.7% and shortens cluster makespan by 15.7% relative to a strong learning-unaware baseline, demonstrating that uncertainty-aligned scheduling can improve cluster-level efficiency while preserving training correctness. We further validate scalability using a calibrated simulator up to 1024 nodes.
Source: Applied Sciences
DOI: 10.3390/app16083725
Authors: Teh-Jen Sun, Eui-Nam Huh
Institution: Kyung Hee University
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,896
Awards: G27118196, G3269894225, G551384806
OpenAlex ID: W7153257298