PubMed Reasoner: Dynamic Reasoning-based Retrieval for Evidence-Grounded Biomedical Question Answering
Year: 2026
Type: conference-paper
Abstract: Trustworthy biomedical question answering (QA) systems must not only provide accurate answers but also justify them with current, verifiable evidence.Retrieval-augmented approaches partially address this gap but lack mechanisms to iteratively refine poor queries, whereas self-reflection methods kick in only after full retrieval is completed.In this context, we introduce PubMed Reasoner, a biomedical QA agent composed of three stages: self-critic query refinement evaluates MeSH terms for coverage, alignment, and redundancy to enhance PubMed queries based on partial (metadata) retrieval; reflective retrieval processes articles in batches until sufficient evidence is gathered; and evidence-grounded response generation produces answers with explicit citations.PubMed Reasoner with a GPT-4o backbone achieves 78.32% accuracy on Pub-MedQA, slightly surpassing human experts, and showing consistent gains on MMLU Clinical Knowledge.Moreover, LLM-as-judge evaluations prefer our responses across: reasoning soundness, evidence grounding, clinical relevance, and trustworthiness.By orchestrating retrieval-first reasoning over authoritative sources, our approach provides practical assistance to clinicians and biomedical researchers while controlling compute and token costs. 1 2 https://github.com/swiftzhang125/ACL_2026_ PubMed_Reasoner, including code, prompt templates, LLM-as-judge evaluation, and baseline systems.
Authors: Yiqing Zhang, Xiaozhong Liu, Fabricio Murai
Institution: Worcester Polytechnic Institute
Language: English
Subfield: Molecular Biology
Domain: Life Sciences
Open Access status: hybrid
OpenAlex ID: W7166849214