Work

Improving nonparameteric Bayesian inference: experiments on unsupervised word segmentation with adaptor grammars

Year: 2009
Type: conference-paper
Abstract: One of the reasons nonparametric Bayesian inference is attracting attention in computational linguistics is because it provides a principled way of learning the units of generalization together with their probabilities. Adaptor grammars are a framework for defining a variety of hierarchical nonparametric Bayesian models. This paper investigates some of the choices that arise in formulating adaptor grammars and associated inference procedures, and shows that they can have a dramatic impact on performance in an unsupervised word segmentation task. With appropriate adaptor grammars and inference procedures we achieve an 87% word token f-score on the standard Brent version of the Bernstein-Ratner corpus, which is an error reduction of over 35% over the best previously reported results for this corpus.
Language: English
Open Access status: bronze
OpenAlex ID: W2126449874
FWCI: 9.013
Cites: 27
Cited by: 151
Related to: 9