Memory Mechanisms for Agentic Systems in Adaptive Automation: A Comparative Evaluation of Recent Advances (2021–2026)
Year: 2026
Type: article
Abstract: Agentic systems operating in adaptive automation environments require persistent memory to retain past actions, qualify outcomes, and improve performance over time. Existing memory mechanisms are architecturally diverse but evaluated in isolation using inconsistent benchmarks, limiting direct comparison. This scoping review systematically examines twelve state-of-the-art memory systems across four architectural categories: Structured Memory, Long-Term Memory, Short and Long-Term Memory, and Workflow and Procedural Memory. Eleven of the twelve are evaluated empirically across nine benchmark datasets spanning conversational question answering, long-document comprehension, and agentic task execution. Memory3 is surveyed but not evaluated, because its memory is built into the model through pre-training and cannot be run on a shared frozen backbone. Every system is evaluated on the datasets aligned to its architectural category, using the same GPT-4o-mini backbone, the same metrics, and the same decoding settings. We also run the backbone with no memory at all, which gives a floor against which each system’s contribution can be measured. Results show that no single architecture dominates all task types. Mem1 leads Contains-Match on conversational tasks while MemoryOS leads Token F1 on the same group, ReadAgent leads multiple-choice long-document comprehension and A-Mem leads open-ended long-document generation, and AWM beats AgentFly on all workflow-level tasks. Measured against the no-memory floor, memory helps a great deal on long-document benchmarks but adds little on multi-hop factoid retrieval, where several systems actually score below the backbone answering from its own pre-trained knowledge. We also identify a verbosity confound. Contains-Match rewards systems that write long answers, with a Spearman correlation of +0.727 between response length and the gap between Contains-Match and Token F1, and it penalizes short exact answers when the reference string carries a trailing period or article. We recommend reporting Token F1 alongside Contains-Match in future evaluations. All evaluation code and datasets are publicly available; see the Data Availability Statement.
Source: AI
DOI: 10.3390/ai7110476
Institution: Worcester Polytechnic Institute
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,930
OpenAlex ID: W7225795490