Product
Help
Pricing
About
Jobs
Log in
Get started
Keyword
Add to collection
key-value cache compression
Description:
Techniques for reducing the memory or storage needed for attention key-value caches during large language model inference.
Primary topic:
Topic Modeling (34%)
Related topics:
Topic Modeling (34%),
Advanced Neural Network Applications (19%),
Natural Language Processing Techniques (12%),
Multimodal Machine Learning Applications (8%)
OpenAlex ID:
keywords/key-value-cache-compression
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
2024
·
DeepSeek-AI
, Liu, Aixin
, et al.
·
arXiv (Cornell University)
·
1,976
PDF
SnapKV: LLM Knows What You are Looking for Before Generation
2024
·
Yuhong Li
, Yingbing Huang
, et al.
·
neural information processing systems
·
1,299
PDF
DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding
2024
·
Wu, Zhiyu
, Chen, Xiaokang
, et al.
·
arXiv (Cornell University)
·
987
PDF
GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
2024
·
Kang, Hao
, Zhang, Qingru
, et al.
·
arXiv (Cornell University)
·
237
PDF
Transformers are Multi-State RNNs
2024
·
Matanel Oren
, Michael Hassid
, et al.
·
Conference on Empirical Methods in Natural Language Processing (EMNLP)
·
206
PDF
LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference
2024
·
Zhongwei Wan
, Ziang Wu
, et al.
·
Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
·
152
PDF
No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization
2024
·
Yang, June Yong
, Kim, Byeongwook
, et al.
·
arXiv (Cornell University)
·
145
PDF
Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent
2024
·
Sun, Xingwu
, Chen, Yanfeng
, et al.
·
arXiv (Cornell University)
·
141
PDF
Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference
2024
·
Nawrot, Piotr
, Łańcucki, Adrian
, et al.
·
arXiv (Cornell University)
·
118
PDF
ClusterKV: Manipulating LLM KV Cache in Semantic Space for Recallable Compression
2025
·
Guangda Liu
, Chengwei Li
, et al.
·
ACM/IEEE Design Automation Conference (DAC)
·
99
PDF
Eigen Attention: Attention in Low-Rank Space for KV Cache Compression
2024
·
Utkarsh Saxena
, Gobinda Saha
, et al.
·
Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
·
99
PDF
Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning
2024
·
Fu, Yu
, Cai, Zefan
, et al.
·
arXiv (Cornell University)
·
90
PDF
Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference
2024
·
Dong, Harry
, Yang, Xinyu
, et al.
·
arXiv (Cornell University)
·
77
PDF
StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding
2025
·
Yang, Yanlai
, Zhao, Zhuokai
, et al.
·
arXiv (Cornell University)
·
76
PDF
LiveVLM: Efficient Online Video Understanding via Streaming-Oriented KV Cache and Retrieval
2026
·
Zhenyu Ning
, Guangda Liu
, et al.
·
ACM/IEEE Design Automation Conference (DAC)
·
69
PDF
Twilight: Adaptive Attention Sparsity with Hierarchical Top-$p$ Pruning
2025
·
Chaofan Lin
, Jiaming Tang
, et al.
·
neural information processing systems
·
64
PDF
ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsification
2024
·
He, Yefei
, Chen, Feng
, et al.
·
arXiv (Cornell University)
·
64
PDF
KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing
2024
·
Yang, Yifei
, Cao, Zouying
, et al.
·
arXiv (Cornell University)
·
57
PDF
DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compaction
2025
·
Yanqi Zhang
, Yuwei Hu
, et al.
·
ACM SIGOPS Symposium on Operating Systems Principles (SOSP)
·
56
FINCH: Prompt-guided Key-Value Cache Compression for Large Language Models
2024
·
Giulio Corallo
, Paolo Papotti
·
Transactions of the Association for Computational Linguistics
·
30
SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation
2025
·
Jialong Wu
, Zhenglin Wang
, et al.
·
Proceedings of the conference - Association for Computational Linguistics. Meeting
·
29
PDF
Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Models
2026
·
Ji, Yicheng
, Zhong, Zhizhou
, et al.
·
arXiv (Cornell University)
·
27
PDF
Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Eviction
2026
·
Kim, Jang-Hyun
, Han, Dongyoon
, et al.
·
arXiv (Cornell University)
·
25
PDF
SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget
2024
·
Wang, Zihao
, Cui, Bin
, et al.
·
arXiv (Cornell University)
·
24
PDF
KV Cache Compression for Inference Efficiency in LLMs: A Review
2025
·
Yanyu Liu
, Jingying Fu
, et al.
·
International Conference on Artificial Intelligence and Intelligent Information Processing (AIIIP)
·
22
Show more works
View as search
Works count:
208
Citation count:
6,710
Year
Topic
Type