TUPOI: Attention-Free Post-Transformer with Constant O(1) Memory
Year: 2026
Type: preprint
Abstract: While standard Transformer architectures (GPT, Claude) suffer from quadratic complexity O(N^2) during training and exploding Key-Value caches during generation, we introduce TUPOI — a foundational attention-free architecture that operates with strictly constant O(1) memory (64.00 KB) and multi-head Delta-KV associative memory. TUPOI represents sequence tokens as canonical position-momentum coordinates (q, p) governed by a 2nd-order Velocity-Verlet leapfrog integrator that strictly preserves Liouville phase volume (det(J) = 1.000000). On the Multi-Query Associative Recall (MQAR) benchmark, TUPOI achieves 100.0% accuracy (surpassing a matched causal transformer baseline at 24.4%) while achieving a 158.8x generation speedup at 32,768 context length. Official repository and code: https://github.com/starface77/TUPOI
Author: Vladov, Danil
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7203708991
Cites:
Cited by:
Related to: