Year: 2026
Type: conference-paper
Abstract: The MedCPT model has demonstrated that strong biomedical retrievers can be trained using proprietary PubMed search logs.In this work, we study whether freely available citation sentences are sufficient to train similarly effective models.We construct a large-scale training dataset of ~62 million citation sentenceabstract pairs extracted from PubMed Central.We train a lightweight BERT-based retrieverreranker model called CiteRec on this dataset and evaluate it across three benchmark settings: (a) the biomedical subset of BEIR for information retrieval, (b) SciRepEval for generalizable scientific document embeddings, and (c) Ci-tancePlus, a new set of ~90 thousand citation sentence-abstract pairs for PubMed-scale citation recommendation.We show that CiteRec performs competitively with MedCPT on the biomedical BEIR subset and outperforms it on SciRepEval.On Citance-Plus, CiteRec achieves strong performance for citation recommendation over the full PubMed corpus, outperforming both MedCPT and a substantially larger Qwen3-Embedding-8B retriever.
Authors: Xing David Wang, Duy Le Thanh, Ulf Leser
Institution: Humboldt-Universität zu Berlin
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Funder: Deutsche Forschungsgemeinschaft
Award: G5091695168
OpenAlex ID: W7166878997