Binary Token Memory: A Scalable Compression Framework for Efficient LLM Inference
Year: 2025
Type: article
Abstract: This work presents the updated and fully binary-based evolution of our previously published Visual Token Memory concept. While the original study explored visual encoding via PNG containers, this paper formalizes an efficient Zstd-compressed binary token memory format designed for scalable LLM applications. This paper presents a lightweight and efficient binary container format for storing and reusing tokenized representations of LLM conversation histories.Unlike prior approaches such as image-based Visual Token Memory (VTM v1), the new Binary Token Memory (VTM v2) leverages Zstandard-compressed raw token arrays with minimal headers.The result is a scalable, GPU-compatible, and persistent memory format that eliminates re-tokenization latency while preserving exact model input structure.Real-world benchmarks conducted on a modest Intel i5 system demonstrate significant speed and storage improvements over traditional JSON or PNG formats.The system supports batch encoding/decoding, precise validation, and future extensions for encrypted or multimodal memory.Aimed at improving production LLM throughput, this method is defensively published to protect prior art and is shared under a CC-BY-NC 4.0 license.
Author: Bagdatli, Cem
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6949637849
FWCI:
Cites:
Cited by:
Related to: