Work

Corpus structure, language models, and ad hoc information retrieval

Year: 2004
Type: conference-paper
Abstract: Most previous work on the recently developed language-modeling approach to information retrieval focuses on document-specific characteristics, and therefore does not take into account the structure of the surrounding corpus. We propose a novel algorithmic framework in which information provided by document-based language models is enhanced by the incorporation of information drawn from clusters of similar documents. Using this framework, we develop a suite of new algorithms. Even the simplest typically outperforms the standard language-modeling approach in precision and recall, and our new interpolation algorithm posts statistically significant improvements for both metrics over all three corpora tested.
Institution: Cornell University
Language: English
Open Access status: green
OpenAlex ID: W2027445772
FWCI: 6.561
Cites: 23
Cited by: 189
Related to: 10