Work

The Entropy of Floating-Point Numbers

Year: 2026

Type: preprint

Abstract: Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a uniformly quantized random variable. Our approximation uncovers a different quantity that provides this link for floating-point quantization. Additionally, we prove that the entropy of a floating-point quantized random variable is approximately unchanged under scaling. Closed-form expressions for the floating-point entropy of common distributions are provided and compared to exact results.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2605.11546

Authors: Daniels, Sultan M., D'Ambrosia, Samuel H., DeWeese, Michael R., Sahai, Anant

Institution: University of California, Berkeley

Language: English

Topic: Numerical Methods and Algorithms

Subfield: Computational Theory and Mathematics

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W7161068793

Cites: 2

Cited by:

Related to: 10