The Length Complexity of the Two-Layer C-RASP Fragment of Chen, Ma and Li Is Θ(KT³) for 2 ≤ K ≤ T
Year: 2026
Type: preprint
Abstract: Transformers trained on short inputs often fail on longer ones. Chen, Ma and Li asked how long training inputs must be: the length complexity of a class is the least N such that an ideal learner shown all labeled strings of length at most N identifies every target exactly. For a two-layer fragment of the attention-like language C-RASP, with K heads and integer parameters of size at most T, they proved TO(K) and guessed Ω(TK) for small K. We show that the length complexity is Θ(KT3) for T ≥ 4 and 2 ≤ K ≤ T: programs that compute different functions differ on an input of length O(KT3), and explicit pairs show this is tight. The proof steers the two programs' scores, a planar walk, into the set where exactly one is positive, using an input of at most three phases. Consequently, their minimum-complexity interpolator needs length O(α3 log α) for targets of complexity at most α, and their guess fails for every fixed K ≥ 4 and large T. The record includes the paper, its LaTeX source, and the code and certificates that reproduce every computed result.
Authors: Baliga, Kundan, Xie, Aiden, Byju, Aditya
Language: English
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225466140