CIRA: Cascaded Implicit Recurrent Attention
Year: 2026
Type: preprint
Abstract: CIRA provides a structured alternative to conventional transformer attention: semantic state formation precedes contextual state formation, and attention operates over dynamically generated equilibrium representations rather than independently parameterized key-value memories. The architecture offers an interpretable state-machine decomposition in which semantic and contextual dynamics, fixed-point representations, and token prediction can be examined separately. This provides a framework for structured implicit dynamics and entropy-regularized attention to improve the semantic coherence, efficiency, and auditability of language models.
Author: Gary Nan Tie
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W7212253566
Cites:
Cited by:
Related to: