Universal Compression at Pragmatic Rates
Year: 2026
Type: conference-paper
Abstract: We consider the problem of optimal universal lossless data compression subject to a constraint on the excess-rate probability. In the non-universal setting, in the regime where the excess-rate probability is required to be below a fixed ϵ > 0, the best asymptotic rate that can be achieved by one-to-one codes on memoryless sources is given by the source entropy. More accurate approximations relying on the central limit theorem provide sharper characterizations of the optimal rate at finite blocklengths. But when the constraint ϵ on the excess-rate probability is small, recent work showed that these approximations fail to be informative and can be quite inaccurate. In the small-excess-rate-probability regime, the best achievable asymptotic rate is given by the inverse of the error-exponent function, and an accurate approximation at finite blocklengths was obtained based on the central limit theorem and large deviations techniques. In this work, we show that, in this small-excess-rate-probability regime, the same performance as when the source distribution is known can be obtained universally over the class of all memoryless sources on a fixed alphabet of size m, at a cost of no more than $\left({\frac{{m - 2}}{2}}\right)\log n$ bits, where n denotes the blocklength. The main new technical tool in the derivation of this result is a fine combinatorial estimate on the number of strings with empirical entropy below a given threshold, which may be of independent interest.
Institution: University of Cambridge
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7204160610