Work

DKITNLP at ArchEHR-QA 2025: A Retrieval Augmented LLM Pipeline for Evidence-Based Patient Question Answering

Year: 2025

Type: conference-paper

Abstract: This paper describes our submission for the BioNLP ACL 2025 Shared task on grounded Question Answering (QA) from Electronic Health Records (EHRs).The task aims to automatically generate answers to patients' health related questions that are grounded in the evidence from their clinical notes.We propose a two stage retrieval pipeline to identify relevant sentences to guide response generation by a Large Language Model (LLM).Specifically, our approach uses a BioBERT based bi-encoder for initial retrieval, followed by a reranking step using a fine-tuned cross-encoder to enhance retrieval precision.The final set of selected sentences serve as an input to Mistral 7B model which generates answers through few-shot prompting.Our approach achieves an overall score of 31.6 on the test set, outperforming a substantially larger baseline model LLaMA 3.3 70B (30.7), which demonstrates the effectiveness of retrieval-augmented generation for grounded QA.

Source: Workshop on Biomedical Language Processing (BioNLP)

DOI: 10.18653/v1/2025.bionlp-share.20

Authors: Provia Kadusabe, Abhishek Kaushik, Fiona Lawless

Language: English

Topic: Machine Learning in Healthcare

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

SDG: Good health and well-being

Open Access status: hybrid

Funders: European Commission, European Regional Development Fund

Award: G7507144715