Work

Memory in AI systems: who writes it, who shares it, and what it cannot forget

Year: 2026
Type: preprint
Abstract: AI systems keep information beyond a single input in their weights, their context window, the caches of their serving systems, retrieval stores, the long-term memory of agents and assistants, the state of recurrent layers, and the hardware that holds all of these. This paper asks how these memories differ and how each can be made to leak, be corrupted or be turned against the system and its users. It compares seven kinds of memory on eight properties and reduces their attack surface to four: who can write to a memory, whom it is shared with, how long it keeps what it holds, and whether it can be made to forget. Four analyses test the comparison: an evidence map of 165 rows from 158 primary studies of attacks, defences and measured properties of memory, coded by memory, threat model and evidence level; the documented memory controls of 26 deployed assistants, frameworks, prompt caches and serving systems; 740 vulnerability records, selected from 2,077, and 50 incidents; and an inventory of 59 benchmarks. Attacks have been demonstrated on deployed products for every kind of memory except recurrent state, and their reach follows the sharing of each memory. In 7 of 8 assistants the model writes memory without being asked, in prompt caches and frameworks isolation is drawn at the vendor's customer rather than at the end user, and several assistants keep the memories derived from a chat after the chat is deleted. Of 114 study rows that test a defence, 38 test it against an adaptive attacker or prove a certified bound, and no benchmark covers serving caches or the hardware. The defences that held best under adaptive attack keep untrusted data out of the control flow or tie memory records to their sources instead of filtering content. The paper derives design principles from these findings: authorized writes, memory kept out of the instruction channel, isolation per end user, and deletion that follows every derived copy.
Language: English
Open Access status: green
OpenAlex ID: W7233706826
Cites:
Cited by:
Related to: 10