Compressed Token Memory: Visual Encoding for Efficient LLM Inference
Year: 2025
Type: article
Abstract: This work introduces a novel token memory system that encodes transformer token sequences into 16-bit grayscale PNG images, addressing the pervasive inefficiency of repetitive dialogue history tokenization in large language models. The proposed method achieves a 2.93-fold reduction in storage requirements compared to uncompressed text while maintaining full compatibility with the Hugging Face ecosystem. By converting token IDs into a GPU-ready visual format, the system eliminates redundant preprocessing steps that currently consume 30-40% of inference time in conventional pipelines. Zero-Copy GPU Access Large Language Models (LLMs) Efficient Inference Memory Optimization Sustainable AI Deployment Transformer Architecture Persistent Dialogue Memory Preprocessing Bottleneck Elimination Language Model Scalability AI Storage Efficiency Compressed Token Encoding DeepSeek Tokenizer Integration
Author: Cem, Bagdatli
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6930821672
FWCI:
Cites:
Cited by:
Related to: