Description: The key-value cache stores attention keys and values from prior tokens to accelerate autoregressive inference in transformer language models.
2025 · Ziqi Pang, Tianyuan Zhang, et al. · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 97