Entropy of a Zipfian Distributed Lexicon
Year: 2013
Type: preprint
Abstract: This article presents the calculation of the entropy of a system with Zipfian distribution. It shows that a communication system tends to present an exponent value close to, but greater than one. This choice both maximizes entropy and, at the same time, enables the retention of a feasible and growing lexicon. This result is in accordance with what is observed in natural languages and with the balance between the speaker and listener communication efforts. On the other hand, the entropy of the communicating source is very sensitive to the exponent value as well as the length of the observable data. Slight deviations on these parameters might lead to very different entropy measurements. A comparison of the estimation proposed with the entropy measure of written texts yields errors in the order of 0.3 bits and 0.05 bits for non-smoothed and smoothed distributions, respectively.
Source: arXiv (Cornell University)
Language: English
Subfield: Theoretical Computer Science
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2963931980