Modus_X 2.0.0: Coordinating Bounded Associative Memory with Recurrent Computation
Year: 2026
Type: preprint
Abstract: Modus_X 2.1.0 is an experimental research release on coordinating bounded associative matrix memory with recurrent vector computation. It separates two evidence leads: MemoryFeedbackArchive for byte-level language modeling and CurrentArchiveDelta for controlled versioned-memory evaluation. CurrentArchiveDelta maintains shared-address current and archive matrices with independently gated update timescales. MemoryFeedbackArchive compresses their mixed retrieval through a low-rank projection and gates it into the vector-stream input before recurrence. This specific coordination pattern is the architectural contribution; fast weights, delta-rule updates, recurrent memory, and hybrid sequence models have substantial prior art. At a fixed 102.4M-character enwik8 budget, MemoryFeedbackArchive improves strongly from 47.44M to 81.49M parameters, then saturates from 81.49M to 99.44M. After matched late annealing, the promoted 81.49M model reaches 1.382445 dense test BPC at 163.84M characters, narrowly improving Modus_X v1.1.1 with 1.54% fewer parameters. Official Mamba remains better on the matched dense endpoint, while MemoryFeedback remains better than the tested official xLSTM. In controlled equal-state experiments, CurrentArchiveDelta outperforms a truncated Transformer KV baseline when the cache cannot retain full context, while full-context KV remains superior. The language-modeling and controlled-memory advantages have not yet been demonstrated in one trained checkpoint. The 1.1 BPC target and trained 1B quality remain open. The archive includes source code, experimental protocols, dense evaluation evidence, controlled-memory studies, rejected interventions, systems-readiness appendices, a restructured whitepaper, model card, provenance and reproducibility metadata, and an integrity manifest.
Author: Chaudhary, Sanyam
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: Google
OpenAlex ID: W7162772486
Cites:
Cited by:
Related to: