Work

MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations

Year: 2026

Type: conference-paper

Abstract: We present MTRAG-UN, a benchmark for exploring open challenges in multi-turn retrieval augmented generation, a popular use of large language models.We release a benchmark of 666 tasks containing over 2,800 conversation turns across 6 domains with accompanying corpora.Our experiments show that retrieval and generation models continue to struggle on conversations with UNanswerable, UNderspecified, and NONstandalone questions and UNclear responses.Our benchmark is available at https://github.com/ IBM/mt-rag-benchmark

Source: Findings of the Association for Computational Linguistics: ACL (Findings of ACL)

DOI: 10.18653/v1/2026.findings-acl.503

Authors: Sara Rosenthal, Yannis Katsis, Vraj Shah, Lihong He, Lucian Popa, Marina Danilevsky

Institution: IBM (United States)

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

OpenAlex ID: W7166837188

FWCI:

Cites:

Cited by:

Related to: 10