Work

Entropy and Inference, Revisited

Year: 2002

Type: book-chapter

Abstract: We study properties of popular near-uniform (Dirichlet) priors for learning undersampled probability distributions on discrete nonmetric spaces and show that they lead to disastrous results. However, an Occam-style phase space argument expands the priors into their infinite mixture and resolves most of the observed problems. This leads to a surprisingly good estimator of entropies of discrete distributions.

Source: The MIT Press eBooks

DOI: 10.7551/mitpress/1120.003.0065

Authors: Ilya Nemenman, Fariel Shafee, William Bialek

Institutions: University of California, Santa Barbara, Kavli Institute for Theoretical Physics, Princeton University

Language: English

Topic: Bayesian Methods and Mixture Models

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: National Science Foundation, Strong

Award: Institute for Theoretical Physics