Empirical study of dense, hybrid and aggregated retrieval methods for scientific expertise recommendation
Year: 2026
Type: dissertation
Abstract: Natural Language Processing (NLP) is one of the most active and widely researched fields in Artificial Intelligence. In particular, embedding models have gained significant popularity and are extensively used across various domains and tasks. Among the different types of embedding models, domain-specific embedding models, those fine-tuned for particular tasks or domains, have recently begun to receive more attention. In this work, we focus on scientific domain-specific embedding models, which have been fine-tuned for scientific text (e.g., SciBERT, SPECTER2, and SciNCL). Despite extensive research on general-purpose embedding models, scientific domain-specific embedding models remain underexplored for many downstream tasks. This thesis investigates and compares the performance of general-purpose and scientific domain-specific embedding models on the task of recommending suitable thesis supervisors. Given a student's thesis and the publication records of professors at the Facultat d'Informàtica de Barcelona (FIB), relevant scientific publications are retrieved and potential supervisors are ranked. The main objectives are to determine which embedding models most effectively represent scientific texts for thesis-to-publication retrieval, identify the best aggregation strategies for ranking professors, and evaluate whether hybrid retrieval techniques improve recommendation quality compared to pure dense retrieval. A series of experiments was conducted to evaluate different embedding models on the thesis-to-publication retrieval task. For each model, the concatenation of the title and abstract of both student theses and professors' scientific publications was embedded. For each thesis embedding, the most similar publication embeddings were retrieved, and professors were ranked using multiple aggregation strategies. A retrieval was considered successful if the actual supervisor of a thesis appeared among the top-ranked professors. This process was repeated across different embedding models, aggregation strategies, and hyperparameters. An additional experiment was conducted in which the pure dense retrieval step was replaced with hybrid retrieval. The results showed that general-purpose embedding models consistently outperformed scientific domain-specific models in retrieval effectiveness on nDCG@10. The performance of the aggregation strategies varied considerably depending on the chosen embedding model and hyperparameter settings. Hybrid retrieval techniques provided small but statistically significant improvements for some models compared to their pure dense retrieval counterparts.
Author: Caballero Castro, Joan
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7164779588
