The Barrier Spin Crossover: Throughput and Energy Cost of Spin-Wait Thresholds in CPU LLM Inference
Year: 2026
Type: software
Abstract: Measurements of how the thread-barrier spin-wait threshold affects CPU LLM inference throughput and energy, on a Raspberry Pi 5 (four homogeneous Cortex-A76 cores, board-level PMIC energy) and on an Intel i7-12700H (heterogeneous Golden Cove plus Gracemont cores). The spin threshold is swept across seven decades. Prefill is insensitive to it while decode is not, and the optimum inverts between an idle machine and a contended one, so no single compile-time constant serves both. The loss is asymmetric: spinning too little costs at most a few percent, spinning too much costs an order of magnitude. Under contention a long spin draws less power and still spends over ten times the energy per token, because the run takes far longer. Includes every raw llama-bench record, the PMIC power traces, the campaign runners, and a script that regenerates every published table from the raw files.
Author: Jacob, Manu Nicholas
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7204659816