Work

Can LLMs Directly Retrieve Passages for Answering Questions from Qur’an?

Year: 2025

Type: conference-paper

Abstract: The Holy Qur'an provides timeless guidance, addressing modern challenges and offering answers to many important questions.The Qur'an QA 2023 shared task introduced the Qur'anic Passage Retrieval (QPR) task, which involves retrieving relevant passages in response to questions written in modern standard Arabic (MSA).In this work, we evaluate the ability of seven large language models (LLMs) to retrieve relevant passages from the Qur'an in response to given questions, considering zero-shot and several few-shot scenarios.Our experiments show that the best model, Claude, significantly outperforms the state-of-the-art QPR model by 28 points on MAP and 38 points on MRR, exhibiting an impressive improvement of about 113% and 82%, respectively.

Source: Arabic Natural Language Processing Workshop (WANLP)

DOI: 10.18653/v1/2025.arabicnlp-main.16

Authors: Sohaila Eltanbouly, Salam Albatarni, Shaimaa Hassanein, Tamer Elsayed

Institution: Qatar University

Language: English

Topic: Information Retrieval and Search Behavior

Subfield: Information Systems

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: Qatar Foundation, Fonds National de la Recherche Luxembourg, Qatar National Research Fund

OpenAlex ID: W4416037684