Greedy Byte-Pair Encoding Achieves the Tight Constant (1-e^{-2})/2 Against the Best Merge Sequence with Short Tokens
Year: 2026
Type: preprint
Abstract: Byte-Pair Encoding (BPE), widely used to build language-model vocabularies, repeatedly merges a most frequent adjacent pair of symbols. No constant above (1-e-2)/2 ≈ 0.432 can bound, on every input, the fraction of the optimal saving that greedy BPE achieves, and the best known general lower bound, 0.391, is from a companion paper. We prove that greedy achieves at least (1/2)(1-(1-2/k)k) ≥ (1/2)(1-e-2) of the optimum whenever some optimal sequence of k merges (partial merges allowed) builds only tokens of at most seven letters. Unconditionally, greedy achieves the same fraction of the best k merges whose tokens have at most seven letters; against this capped optimum the limiting constant is best possible, by Yang's construction, whereas for the conditional result we do not claim tightness. Both results follow from a per-step inequality, OPT ≤ 2Gi + k gi+1, where Gi is greedy's saving after i merges and gi+1 its next saving; its proof keeps, for each rule of the optimum, its merge points at one pair of symbols of greedy's current string. The proof is analytic for tokens of at most five letters and partly computer-assisted for six and seven. These results appeared, with proofs, in versions 1.0-1.2 of the authors' preprint doi:10.5281/zenodo.24023719 (now restructured); this is their self-contained account. The record includes the paper, its LaTeX source, and the code and outputs of the exhaustive computations used in the proofs; each finite lemma for token lengths six and seven is repeated by a second, separately written program.
Authors: Baliga, Kundan, Xie, Aiden, Byju, Aditya
Language: English
Subfield: Theoretical Computer Science
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225588372