Work

SparseEmbed: Learning Sparse Lexical Representations with Contextual Embeddings for Retrieval

Year: 2023
Type: conference-paper
Abstract: In dense retrieval, prior work has largely improved retrieval effectiveness using multi-vector dense representations, exemplified by ColBERT. In sparse retrieval, more recent work, such as SPLADE, demonstrated that one can also learn sparse lexical representations to achieve comparable effectiveness while enjoying better interpretability. In this work, we combine the strengths of both the sparse and dense representations for first-stage retrieval. Specifically, we propose SparseEmbed - a novel retrieval model that learns sparse lexical representations with contextual embeddings. Compared with SPLADE, our model leverages the contextual embeddings to improve model expressiveness. Compared with ColBERT, our sparse representations are trained end-to-end to optimize both efficiency and effectiveness.
Language: English
Open Access status: hybrid
OpenAlex ID: W4384890771
FWCI: 5.685
Cites: 26
Cited by: 41
Related to: 10