RAGonauts at SemEval-2026 Task 8: BM25 Retrieval with Last-Turn Query Formulation for Multi-Turn RAG Conversations
Year: 2026
Type: conference-paper
Abstract: This paper describes the submission to Task A of SemEval-2026 Task 8: MTRAGEval which evaluates passage retrieval for multi-turn Retrieval-Augmented Generation (RAG) conversations across multiple knowledge domains.The task requires retrieving relevant supporting passages given conversational history, where user queries often contain implicit references and incomplete contextual information.This paper proposes a lightweight and training-free retrieval framework based on BM25 ranking combined with conversational query formulation.Queries are derived from dialogue turns and retrieval is performed using domainspecific indices to preserve corpus relevance.Without neural retrievers or fine-tuning, our system achieves an nDCG@5 score of 0.2836 on the official evaluation set, ranking 33 rd on the leaderboard.This result demonstrates that sparse lexical retrieval remains an efficient and reproducible baseline for conversational RAG systems.
Institution: Sri Sivasubramaniya Nadar College of Engineering
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166838930