Work

uircis at SemEval-2026 Task 8: A Unified Lightweight Pipeline for Multi-Turn RAG Evaluation

Year: 2026

Type: conference-paper

Abstract: This paper describes a system for SemEval-2026 Task 8 (MTRAGEval), covering Subtask A (retrieval) and Subtask B (generation) (Rosenthal et al., 2026b).We present a replicable multi-turn RAG pipeline design that runs entirely with open-weight models on a single GPU without relying on proprietary APIs, combining (1) query rewriting with Qwen2.5-7B-Instruct,(2) dense retrieval with BGE-M3, and (3) crossencoder reranking with BGE-Reranker-v2-M3.For generation it reuses Qwen2.5-7B-Instruct with strict grounding instructions.On the official test set, the retrieval component achieves nDCG@5 of 0.4422 and the generation component achieves a harmonic mean score of 0.5664.We provide an ablation study quantifying the contributions of rewriting and reranking, and an error analysis guided by the organizers' analytics and answerability classes.

Source: International Workshop on Semantic Evaluation (SemEval)

DOI: 10.18653/v1/2026.semeval-1.394

Authors: Jiaqi Zhang, Wenbin Duan, Yingqi Zhang, Yan Li, Binyang Li

Institution: University of International Relations

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: University of International Relations

OpenAlex ID: W7166796110