tt501 at ArchEHR-QA 2026: Few-Shot Prompting with Retrieval-Augmented Generation for Grounded Clinical EHR Question Answering
Year: 2026
Type: conference-paper
Abstract: We present the system developed by team tt501 for the ArchEHR-QA 2026 shared task, participating in three subtasks: Evidence Identification (Subtask 2), Answer Generation (Subtask 3), and Evidence Alignment (Subtask 4).All approaches rely exclusively on prompt engineering with reasoning-capable large language models from xAI (grok-4-fast-reasoning and grok-4-1-fast-reasoning) via API, requiring no task-specific fine-tuning.For Subtask 2, we propose a two-stage hybrid pipeline combining BM25 keyword retrieval with full-context Chain-of-Thought prompting and a secondary LLM refinement step, achieving a best Strict Micro-F1 of 58.84.For Subtask 3, we implement a retrieval-augmented generation (RAG) approach that chains predicted evidence from Subtask 2 with few-shot prompting to produce grounded 75-word clinical answers, reaching an overall score of 31.36.For Subtask 4, we develop a recall-optimised few-shot prompting strategy with reasoning annotations embedded in exemplars, enabling the model to correctly identify many-to-many evidence-answer alignments and achieving a best Micro-F1 of 79.11.Our results demonstrate that carefully engineered prompts with reasoning-capable LLMs can achieve competitive performance on fine-grained clinical evidence tasks without any task-specific training.
Author: Tai Tan Tran
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166571667