Year: 2026
Type: conference-paper
Abstract: Large language models (LLMs) are designed for discrete tokens, yet they operate in a continuous embedding space.Recent context compression methods exploit this property by encoding text into dense vectors for frozen LLM decoding.However, a key question remains unanswered: how does a frozen LLM interpret continuous vectors that encode complex semantics?We investigate this through controlled reconstruction experiments.Our analysis reveals a critical geometric property: successful compression encoders learn to produce vectors with L2 norms two orders of magnitude higher than standard embeddings.Norm-scaling interventions provide strong evidence that this highnorm regime is an enabling factor for frozen-LLM decoding, while leaving open whether the effect arises from attention dominance, lownorm suppression, or both.Based on this finding, we propose a landmark-based compression framework for long contexts.Our encoder uses bidirectional attention over landmark tokens, which captures global dependencies and avoids semantic fragmentation from segment-based methods.Experiments on text reconstruction and four QA benchmarks provide evidence for our method.At 4x compression, our method is strongest on SQuAD and AdversarialQA and remains competitive on average.
Authors: Yunsheng Zeng, Yongmei Tan
Institution: Beijing University of Posts and Telecommunications
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166874809