Ask What Forgetting Would Cost: A Label-Free Judge Gates Nightly Replay of Free-Text Facts
Year: 2026
Type: preprint
Abstract: Can a model decide what is worth remembering without anyone telling it what matters? We test this on nightly LoRA consolidation, where whatever is not replayed is forgotten. Our earlier work showed that replaying facts by criticality protects them, but on template data and with a probe trained on human labels. Here we remove both. The facts are 150 free-text sentences in Spanish and English, and the selector uses no labels: a 1.5B model is asked once per fact whether forgetting it could cause harm (health, safety, money, legal), and its probability of “Yes” ranks the facts at AUROC 0.992. Over fifteen nights of LoRA consolidation in Qwen2.5-0.5B-Instruct, judge-gated replay retains 19.6 of 38 critical facts, against 8.8 for random replay and 11.0 for replaying what was forgotten, winning in all five seeds (exact McNemar p = 1.2e-8 and 3.3e-6). The gain is largest on implicit critical facts (10.6 of 18 against 3.2), and a keyword selector marks 12 lexical traps as salient where the judge marks none. A replication with Llama-3.2-1B-Instruct, under a rule written before launch, gives 22.0 against 11.7 with no loss of trivial facts. Knowing that a fact was forgotten adds 2.2 facts over chance; knowing what it was worth adds 10.8. All data are LLM-written, the students are small, and every deviation from the written plan is reported. Data, code and full logs are included.
Author: Goin del Rio, Sebastian
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225878032
