L-Dynamic Attention: Learned Age-Aware KV Cache Management for Efficient Long-Context Transformers
Year: 2026
Type: preprint
Abstract: : We introduce L-Dynamic Attention, a learned Key-Value (KV) cache management mechanism designed to enable efficient long-context Transformer inference. In modern Large Language Models (LLMs), processing extended sequences suffers from linear KV cache growth and massive memory bottlenecks. Shifting away from rigid, hand-crafted pruning heuristics, our framework assigns each token a dynamic scalar utility score w_j ∈ [0,1], estimated via a lightweight predictor trained on key embeddings, positional tokens, and local context. These individual utility scores are combined with token age (temporal entropy t_j) into a unified viability function v_j = w_j^2 / (t_j^age + ε), directly instantiating the foundational Lt-parameter framework (v = L²/t) within deep learning architectures. Under an adaptive percentile thresholding eviction policy, non-essential tokens are systematically collapsed to maintain a strict memory budget. Empirical evaluations on LLaMA-2 7B (up to 32k context lengths) demonstrate up to a 10x memory footprint reduction with negligible accuracy degradation (<1.5% perplexity increase on PG19, and <3% drop in long-context retrieval accuracy). We provide a rigorous theoretical interpretation of this mechanism as an approximate solution to a constrained memory optimization problem under an evolutionary survival-process paradigm.
Author: Usychenko, Stanislav
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7161677305
Cites:
Cited by:
Related to: