When Mamba Needs Attention
Year: 2026
Type: preprint
Abstract: We ask whether a linear-time state-space (Mamba) encoder can drive routing decisions as well as a quadratic-time attention encoder on Euclidean TSP-100, under identical POMO reinforcement learning and a byte-identical decoder, with only the encoder swapped and parameter counts matched. The obvious recipe—a Hilbert-curve serialization scanned by a unidirectional Mamba—is killed: at a matched 250k-step budget it reaches a 7.50 ± 1.03 single-trajectory optimality gap over three seeds versus attention's 2.95%. Diagnostics localize the failure to a short, one-directional receptive field; an ordering ablation shows a locality prior confers no benefit for causal scans and roughly doubles seed variance. Repair follows the diagnosis: pooled global channels fail, a param-matched bidirectional scan reaches multistart parity with attention (1.94 ± 0.09 vs 1.91) and cuts seed variance tenfold, and a hybrid inserting one attention layer into a bidirectional-Mamba stack (4.5% fewer parameters) beats attention on multistart tour quality at a converged 500k budget (worst hybrid seed 1.365 < best attention 1.589). All claims are scoped to N=100; the O(N) payoff remains untested. Code, curves, and pre-registered decision rules: github.com/Umaraslam66/serpentine
Author: Aslam, Umar
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7170136438
Cites:
Cited by:
Related to: