Token Write Amplification in LLM-Mediated State Stores
Year: 2026
Type: conference-paper
Abstract: LLM-based applications increasingly rely on persistent state stores to maintain durable memories across long-running interactions. Unlike conventional storage systems, these stores often invoke LLMs on the write path to transform raw context into semantic state, shifting maintenance cost from bytes written to tokens processed. We introduce token write amplification (TWA), the ratio of write path LLM tokens to raw context tokens written, to quantify this byte-invisible cost. Across five state store configurations and seven agent workloads, native write paths incur an average TWA of 8.00 ×, showing that semantic state maintenance can consume far more tokens than the context being stored. By accounting for writes at the level of semantic units, we find that 54.9% of native TWA is associated with semantics already covered by the existing state, indicating that redundant semantic processing is a substantial write path cost. Motivated by this observation, we propose a lightweight, backend-agnostic semantic coalescing buffer that groups related writes before storage. Across the same workloads, semantic coalescing reduces average TWA from 8.00 × to 3.02 × without backend modifications, demonstrating that semantic overlap is a key optimization target for LLM-mediated state stores.
Institution: University of Science and Technology of China
Language: English
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Award: G8551023727
OpenAlex ID: W7213977751