Get started
Work

Construction before search: a pre-registered test of what an LLM memory builder keeps

Year: 2026
Type: preprint
Abstract: A pre-registered test of whether repairing the construction step of an LLM-extracted agent memory changes how it compares with the raw record. Recent studies compare extracted memory with retrieval over raw conversation and keep the extraction step at a default. This paper varies it on LongMemEval s_cleaned: how much of the conversation the builder sees (the window) and the instructions it extracts under (the criterion). The path to the result is reported with it. The first instrument failed its own registered floor; the first amendment calibrated four readers and selected none; an exploratory run located the larger loss between raw evidence and distilled units; a revised criterion, C2, passed calibration on development data; and the confirmatory sample of 242 instances was defined by provenance after adversarial review. Results. All four registered controls passed. Repairing both factors against repairing neither raised accuracy from 0.153 to 0.818: +66.5 points [+60.6, +72.5] under judge 1 and +66.9 under a second judge family. A length-matched arm rules out its own form of padding. A raw-evidence arm on the same 242, registered by tag before it ran, put C2 units of the evidence sessions about 5 points below the raw turns of those sessions (−5.4 [−10.3, −0.8] and −5.0 [−9.9, 0.0]); judge 1 detects a loss without non-inferiority at 10 points and judge 2 the reverse, so neither reading holds under both. The arm compares two served materials under perfect retrieval and does not test the retention prediction of arXiv 2601.00821. The result confirms a revised intervention, not the originally registered hypotheses. The registration records and the result records with their data are cited in §9 of the paper. Not peer reviewed.
DOI: 10.5281/zenodo.23964959
Language: English
Open Access status: green
OpenAlex ID: W7225223560
Cites: 2
Cited by:
Related to: