Work

Thalamo-koniocortical simulations reveal hardware-oriented principles for Transformer-like architectures

Year: 2026
Type: preprint
Abstract: Transformer architectures process distributed representations through explicit matrix operations, whereas biological circuits compute through local electrical and synaptic dynamics. We ask whether Transformer-like functions can emerge in three separately simulated phenomenological models of a thalamo-koniocortical pathway. The thalamic model converts overlapping sensory activity into less-overlapping components and reconstructs incomplete inputs; the koniocortical model transforms these components into sparse, stable, pattern-selective events; and the cortical model binds ordered events into frame-like representations and temporal sequences. Mathematical analysis links Hebbian retrieval and Transformer attention through similarity-dependent weighting. These correspondences suggest that algorithmic operations may arise from the physical dynamics of interacting units, motivating hardware-oriented and neuromorphic alternatives. Although simulated separately, the models define a testable pathway from sensory recoding to sparse representation, temporal binding and competitive selection.
Language: English
Open Access status: green
OpenAlex ID: W7234733702
Cites: 80
Cited by:
Related to: 10