Adaptive-Depth Recursion And State-Of-The-Art Efficient Sequence Models For The Oral Resistome: A Homology-Aware Benchmark With External Validation
Year: 2026
Type: preprint
Abstract: Background . Resistance genes form an abundant reservoir of resistance factors within the oral microbiome. Scalable attention alternatives may be able to annotate them; however, none has been evaluated for annotating them without learning from memorization of the data. Methods . We curated 26,324 resistome proteins from UniProtKB, CARD, and NCBI’s AMRFinderPlus datasets, clustered by 40% sequence similarity and 80% coverage. The proteins were partitioned into clusters. Eight causal residue-level architectures (Transformer, Funnel-Transformer, Performer, Hyena, causal state-space model, Mixture-of-Recursions MoR with a causal token choice router, parameter-matched MoR, and ablation of pairwise merge gating) were trained with identical data (up to 100 epochs with early stopping) for up to three random seeds and evaluated on held-out clusters and on independent databases with autoregressive causality audit. We used antibiotic class prediction with frozen representations as a measure of transferability. Results . 26,324 proteins aggregated into 1,437 clusters (the largest being 3,333). On held-out clusters, all architectures outperformed the residue composition benchmark (perplexity 17.50), with Hyena outperforming Transformer (14.93 ± 0.58 vs 15.33); MoR (0.79M parameters) achieved 16.37, a sixth of the baseline value. On external validation (against the CARD dataset), only 4 of 8 architectures exceeded the composition benchmark perplexity of 17.99; the Transformer degrades by 27%; constrained architectures did better, with the MoR reaching 17.40, parameter-matched to the Transformer. Perplexity on external validation increased as similarity between the evaluation and training sets increased. MoR is 5.7x faster than Transformer on 4096 residues; the router uses 42% fewer block applications than the fixed depth. Conclusions . Performance of architectures trained on the oral resistome critically depends on the way evaluation is done: homology-aware partitions and external validation reverse the order of architectures seen when evaluated on held-out clusters only. Recursive architectures perform best on external validation and are the least costly to compute, using less than 1/7th of the parameters compared to Transformers. Yet, for proteins without apparent homology to training sets, none of the eight architectures outperform residue composition.
Source: Research Square
Institution: Ajman University
Language: English
Subfield: Microbiology (medical)
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7225872649
