D2.2 Retrieval model for conversational style queries
Year: 2025
Type: article
Abstract: This report presents research and experimental developments under WP2, Task 2, focusing on factual information retrieval (FIR) with conversational queries. We address open challenges including enhancing large language model (LLM) efficiency, mitigating hallucinations, identifying bias, and developing methodologies for semi-automatic creation of FIR training and benchmark datasets. Building upon prior work in T2.1, we extend a speech-to-text processing pipeline that enables reuse of existing text-based FIR systems for conversational speech data, using a BGE-M3 multilingual retrieval model. Two alignment strategies between speech and text embeddings are explored, with input-space alignment yielding superior results. Additionally, we propose a methodology for constructing a multimodal QA dataset from natural podcast conversations, advancing resources for FIR on spontaneous speech. Our work also expands on synthetic dataset creation using Retrieval-Augmented Generation (RAG) techniques, contributing to better contextualization and trustworthiness of LLM-based systems. Further, we investigate document plausibility and the associated findability bias, providing strategies to improve retrieval robustness and diversity. Finally, we explore two emerging research directions: introducing span-copying mechanisms to enhance LLM efficiency and reduce hallucinations and designing fine-grained retrieval models combining LLM supervision with lightweight, interpretable retrieval architectures. These efforts collectively advance multimodal, reliable, and efficient FIR systems suitable for real-world conversational applications.
Authors: Plchot, Oldřich, Fajčík, Martin, Sedláček, Šimon, Valentini, Francisco Tomas, Kesiraju, Santosh
Institution: Brno University of Technology
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: European Commission
OpenAlex ID: W7081924877
FWCI:
Cites:
Cited by:
Related to: