Work

Contrastive estimation

Year: 2005

Type: conference-paper

Abstract: Conditional random fields (Lafferty et al., 2001) are quite effective at sequence labeling tasks like shallow parsing (Sha and Pereira, 2003) and named-entity extraction (McCallum and Li, 2003). CRFs are log-linear, allowing the incorporation of arbitrary features into the model. To train on unlabeled data, we require unsupervised estimation methods for log-linear models; few exist. We describe a novel approach, contrastive estimation. We show that the new technique can be intuitively understood as exploiting implicit negative evidence and is computationally efficient. Applied to a sequence labeling problem---POS tagging given a tagging dictionary and unlabeled text---contrastive estimation outperforms EM (with the same feature set), is more robust to degradations of the dictionary, and can largely recover by modeling additional features.

Source: Annual Meeting of the Association for Computational Linguistics (ACL)

DOI: 10.3115/1219840.1219884

Authors: Noah A. Smith, Jason Eisner

Institution: Johns Hopkins University

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: bronze

Funder: National Science Foundation

Award: ITR: Weighted Dynamic Programming for Statistical Natural Language Processing