Pruning Optimization for Efficient Top-k Document Retrieval with Learned Sparse Representations
Year: 2024
Type: dissertation
Abstract: Efficiently searching for relevant documents on a large dataset typically employs an initial retrieval stage to extract the most relevant candidates. This process often utilizes a sparse data structure known as an inverted index, coupled with a simple yet fast ranking method, to identify the top-k matches for a given query. Recent advancements in retrieval methodologies have seen the integration of learned sparse representations, leveraging transformer-based language models to expand and weigh document terms, thereby enhancing the semantic alignment between query and document vocabularies. However, the distribution of these neural model generated weights differs from traditional term frequency-based BM25, posing challenges for dynamic pruning algorithms for inverted index traversal, and significantly impeding retrieval speed. Moreover, these techniques often exacerbate document representation sparsity, further slowing down retrieval algorithms. Addressing these challenges, this thesis focuses on accelerating document retrieval algorithms through representation sparsification, BM25-guided document pruning, and cluster-based sparse retrieval strategies.
Author: Qiao, Yifan
Institution: University of California, Santa Barbara
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Keywords: BM25, neural language models, retrieval speed
Open Access status: green
Funder: National Science Foundation
OpenAlex ID: W6995889771
