Year: 2025
Type: conference-paper
Abstract: Literature discovery is a critical component of scientific research.Modern discovery systems leveraging Large Language Models (LLMs) are increasingly adopted for their ability to process natural language queries (NLQs).To assess the robustness of such systems, we compile two NLQ datasets and submit them to nine widely used discovery platforms.Our findings reveal that LLM-based search engines struggle with precisely formulated queries, often producing numerous false positives.However, precision improves when LLMs are used not for direct retrieval but to convert NLQs into structured keyword-based queries.As a result, hybrid systems that integrate both LLM-driven and keyword-based approaches outperform purely keyword-based or purely LLM-based discovery methods.
Institutions: University of Zurich, ETH Zurich, Institute of Neuroinformatics
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4412944436