Origin-Paged Memory: Treating Language-Model Context as Virtual Memory A Pre-Registered First Test of the Origin-Paged Attention Program on a 4B-Parameter Model
Year: 2026
Type: preprint
Abstract: Language-model agents that work for hours, or read documents far longer than their context window, need a principled way to decide what stays in context. We study Origin-Paged Attention (OPA), a research program that treats context like virtual memory in an operating system. The archive is split into fixed-size chunks called origins. The program has two layers. Its memory layer, Origin-Paged Memory (OPM), keeps a few origins in working slots, pins exact evidence, and pages an earlier origin back in when an entity becomes relevant late, using an index of the entities each origin mentions. Its attention layer, Origin-Anchored Attention (OAA), would move the same idea inside the model by giving every origin its own positional frame and an address in hyperbolic space. We prove that OAA attention factors exactly into a choice of origin times ordinary attention within it, so the error caused by paging equals the attention mass that was paged out. We then test 13 pre-registered hypotheses on the open Qwen3-4B-Instruct model, with every verdict rule fixed before any test data existed. On synthetic late-binding multi-hop questions at 32K tokens, OPM answered 76.6% correctly, against 62.4% for one-shot retrieval (a gain of 14.2 points, 95% CI 9.2–19.1); it also beat a gist-memory agent by 39.6 points, and pinning cut repeated page-ins by 2.54 per item. A graph of links between origins, hyperbolic distances, and a “red-dot” prefix that carries the end of one origin into the next added nothing measurable. At the attention level, a short fine-tuning run did not teach OAA to attribute facts to their origin (31.2% against 99.6% for standard attention), so the remaining attention-level and systems tests were not run. The memory layer is supported; moving it inside attention remains open.
Source: Preprints.org
Author: Aaditya Rajgor
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7226030409
