Greedy byte-pair encoding asymptotically maximizes token savings on fixed alphabets
Year: 2026
Type: preprint
Abstract: An explicit binary construction at the shortest asymptotic input scale for strict greedy loss, with persistent deficit after a forced first merge.
Authors: Baliga, Kundan, Xie, Aiden, Byju, Aditya
Language: English
Subfield: Theoretical Computer Science
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7235048612
