Ribot: A Language Model That Reads Memory Networks It Never Saw
Year: 2026
Type: article
Abstract: A language model gets new knowledge in two ways: in its context, or by training. The context is limited, and training is expensive and changes the whole model. I introduce Ribot: a language model with a small memory network as one of its parts. The memory network holds data in its own weights, and it is trained on it alone, without the rest of the model. The rest of the model is trained once to read memory networks, with swap training: a hundred or more frozen memory networks are swapped during its training, so the only way to answer is to read the one plugged in. Trained on made-up facts, Ribot reads memory networks it never saw at 92.4% exact, and 0% with the memory removed. Trained on ordinary words, it reads 500 facts from one memory network at 80% to 90% exact, with nothing in its prompt, about as accurate as retrieval (RAG) over the same facts. With the right fact alone in the prompt, the original model, SmolLM2-135M, is still more accurate at this size. The models and code are released. Models and code: https://huggingface.co/aikronus-labs/ribot-135m (version 0.2)
Author: Aikronus
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225791426
FWCI:
Cites:
Cited by:
Related to: