Apollo: Accelerating Load Requests via Multi-Level Cache Miss Load Prediction
Year: 2026
Type: article
Abstract: Accelerating load requests is an effective way to improve processor performance through reducing load request latency. Currently, state-of-the-art methods, including Hermes and TLP, are limited to accelerating off-chip load requests that are correctly predicted and cannot accelerate mispredicted off-chip load requests and any on-chip load requests. To overcome this limitation, we propose a new technique called Apollo that integrates TLP. The key innovation of Apollo lies in its transformation of the perceptron-based off-chip prediction paradigm, pioneered by Hermes, into a comprehensive multi-level cache miss prediction technique. The workflow of Apollo is as follows: (1) predicting whether a load request will miss the L1D or L2, and (2) performing arbitration to decide whether to issue a speculative load request and to which cache level the request is issued, and (3) issuing a speculative load request to the lower-level cache (either L2 or LLC) after arbitration for those predicted to miss the L1D or L2, while allowing the regular load request to concurrently access the cache hierarchy. If the prediction is correct, the regular load request eventually misses the L1D or L2 and waits for the speculative load request to finish. Therefore, Apollo can hide the L1D access latency for correctly predicted L1D miss load requests, and both L1D and L2 access latency for correctly predicted L2 miss load requests. To enable Apollo, we propose a lightweight L1D miss load predictor (L1MP), a lightweight L2 miss load predictor (L2MP), and an arbiter called Athena. L1MP and L2MP predict whether load requests will miss in the L1D and L2, respectively, while Athena performs arbitration to control the issuance of speculative load requests. Our evaluation using a diverse set of workloads shows that Apollo provides a geometric mean (geomean) performance improvement of 15.2% for the single-core processor, outperforming Hermes by 9.6% and TLP by 6.4%. For the multi-core processor, Apollo provides a geomean performance improvement of 18.5%, which is 16.8% higher than Hermes and 5.8% higher than TLP.
DOI: 10.1145/3818683
Authors: Zhengwei Huang, Wei Guo, Yongwen Wang
Institution: National University of Defense Technology
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7162684207