Comparison of different Unique hard attention transformer models by the formal languages they can recognize
Year: 2025
Type: preprint
Abstract: This note is a survey of various results on the capabilities of unique hard attention transformers encoders (UHATs) to recognize formal languages. We distinguish between masked vs. non-masked, finite vs. infinite image and general vs. bilinear attention score functions. We recall some relations between these models, as well as a lower bound in terms of first-order logic and an upper bound in terms of circuit complexity.
Source: arXiv (Cornell University)
Author: Ryvkin, Leonid
Language: English
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4416072350
Cites:
Cited by:
Related to: