Product
Help
Pricing
About
Jobs
Log in
Get started
Keyword
Add to collection
long-context inference
Description:
Long-context inference is the use of a language model to process lengthy inputs and produce predictions or responses efficiently.
Primary topic:
Topic Modeling (35%)
Related topics:
Topic Modeling (35%),
Parallel Computing and Optimization Techniques (15%),
Advanced Neural Network Applications (15%),
Natural Language Processing Techniques (14%)
OpenAlex ID:
keywords/long-context-inference
FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
2025
·
Ye, Zihao
, Chen, Lequn
, et al.
·
arXiv (Cornell University)
·
454
PDF
Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
2025
·
Ruoyu Qin
, Zheming Li
, et al.
·
ACM Transactions on Storage
·
346
PDF
DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
2024
·
Xiao, Guangxuan
, Tang, Jiaming
, et al.
·
International Conference on Learning Representations
·
245
LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism
2024
·
Bingyang Wu
, Shengyu Liu
, et al.
·
ACM SIGOPS Symposium on Operating Systems Principles (SOSP)
·
234
PDF
GNN-RAG: Graph Neural Retrieval for Efficient Large Language Model Reasoning on Knowledge Graphs
2025
·
Costas Mavromatis
, George Karypis
·
Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
·
224
PDF
A Survey on Large Language Model Acceleration based on KV Cache Management
2024
·
Li, Haoyang
, Li, Yiming
, et al.
·
Transactions on machine learning research
·
166
PDF
LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference
2024
·
Zhongwei Wan
, Ziang Wu
, et al.
·
Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
·
152
PDF
LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference
2024
·
Fu, Qichen
, Cho, Minsik
, et al.
·
arXiv (Cornell University)
·
137
PDF
QAQ: Quality Adaptive Quantization for LLM KV Cache
2025
·
Wen Cheng
, Shichen Dong
, et al.
·
IEEE International Conference on Computer Vision Workshops
·
96
PDF
D2O: Dynamic Discriminative Operations for Efficient Long-Context Inference of Large Language Models
2024
·
Wan, Zhongwei
, Wu, Xinjian
, et al.
·
International Conference on Learning Representations
·
87
HyperAttention: Long-context Attention in Near-Linear Time
2023
·
Han, Insu
, Jayaram, Rajesh
, et al.
·
International Conference on Learning Representations
·
81
Squeezed Attention: Accelerating Long Context Length LLM Inference
2025
·
Coleman Richard Charles Hooper
, Sehoon Kim
, et al.
·
Proceedings of the conference - Association for Computational Linguistics. Meeting
·
76
PDF
ThinK: Thinner Key Cache by Query-Driven Pruning
2024
·
Xu, Yuhui
, Jie, Zhanming
, et al.
·
International Conference on Learning Representations
·
68
TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
2025
·
Wei Wu
, Zhuoshi Pan
, et al.
·
Conference on Empirical Methods in Natural Language Processing (EMNLP)
·
58
PDF
Understanding the Differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks
2024
·
Jerome Sieber
, Carmen Alonso
, et al.
·
neural information processing systems
·
53
Compute Or Load KV Cache? Why Not Both?
2024
·
Jin, Shuowei
, Liu, Xueshen
, et al.
·
arXiv (Cornell University)
·
44
PDF
Context Parallelism for Scalable Million-Token Inference
2024
·
Yang, Amy
, Yang, Jingyi
, et al.
·
arXiv (Cornell University)
·
40
PDF
KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference
2025
·
Li, Xing
, Xing, Zeyu
, et al.
·
arXiv (Cornell University)
·
39
PDF
KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark of Long Context Capable Approaches
2024
·
Jiayi Yuan
, Hongyi Liu
, et al.
·
Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
·
39
PDF
QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache
2025
·
Tiwari, Rishabh
, Xi, Haocheng
, et al.
·
arXiv (Cornell University)
·
35
PDF
Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models
2025
·
Shutova, Alina
, Malinovskii, Vladimir
, et al.
·
arXiv (Cornell University)
·
33
PDF
Integrating Artificial Intelligence into Operating Systems: A Survey on Techniques, Applications, and Future Directions
2024
·
Zhang, Yifan
, Zhao, Xinkui
, et al.
·
arXiv (Cornell University)
·
30
PDF
Characterizing Prompt Compression Methods for Long Context Inference
2024
·
Jha, Siddharth
, Erdogan, Lutfi Eren
, et al.
·
arXiv (Cornell University)
·
30
PDF
Beluga: A CXL-Based Memory Architecture for Scalable and Efficient LLM KVCache Management
2026
·
Xinjun Yang
, Qingda Hu
, et al.
·
Proceedings of the ACM on Management of Data
·
27
LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference
2025
·
Wang, Guangtao
, Upasani, Shubhangi
, et al.
·
arXiv (Cornell University)
·
27
PDF
Show more works
View as search
Works count:
558
Citation count:
4,393
Year
Topic
Type