Work

Entropy rate constancy in text

Year: 2001

Type: conference-paper

Abstract: We present a constancy rate principle governing language generation. We show that this principle implies that local measures of entropy (ignoring context) should increase with the sentence number. We demonstrate that this is indeed the case by measuring entropy in three different ways. We also show that this effect has both lexical (which words are used) and non-lexical (how the words are used) causes.

Source: Annual Meeting of the Association for Computational Linguistics (ACL)

DOI: 10.3115/1073083.1073117

Authors: Dmitriy Genzel, Eugene Charniak

Institution: Brown University

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: bronze

Funder: National Science Foundation

Awards: Learning and Action in the Face of Uncertainty: Cognitive, Computational and Statistical Approaches, ITR/SY(CISE) Learning Syntactic/Semantic Information for Parsing, ITR: Information Access to Spoken Documents