Surprise Is Not Importance: Criticality Read from Hidden States Gates Sleep-Like Consolidation
Year: 2026
Type: preprint
Abstract: A model that learns new facts every night forgets what it does not rehearse, so a fixed replay budget is a choice of what matters. Surprise is the usual guide; we show it is the wrong one. A small probe on the frozen residual stream of Qwen2.5-0.5B-Instruct scores how critical a fact is (health, safety, money, legal risk) at AUROC 0.99, while answer-loss surprise scores 0.53 and falls for lexical traps. Used to choose what to replay over ten nights of LoRA consolidation, the probe retains 20.3 of 30 critical facts, against 12.7 for random replay and 10.3 for surprise, on par with a label oracle. The signal is semantic, not lexical: on an independent bilingual exam outside the training templates the probe keeps 0.98–0.99 while word TF-IDF drops to 0.65. Two limits are reported: off-the-shelf sentence encoders match the probe, and criticality that depends on facts stored in the adapter is read only weakly (0.57). All data are synthetic or LLM-written; three-seed contrasts are descriptive.
Author: Goin del Rio, Sebastian
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225804221
Cites:
Cited by:
Related to: