N&N at QIAS 2025: Chain-of-Thought Ensembles with Retrieval-Augmented framework for Classical Arabic Islamic
Year: 2025
Type: conference-paper
Abstract: We present our system developed for the Question-and-Answer in Islamic Studies Assessment Shared Task on Evaluating LLMs for Islamic Knowledge (QIAS 2025), which focuses on answering Arabic multiple-choice questions (MCQs) derived from classical Islamic texts.Our methodology integrates fewshot chain-of-thought prompting across multiple LLMs, enhanced by a majority-vote ensemble mechanism.In situations of ensemble uncertainty, we deploy a retrieval-augmented re-prompting module that extracts contextually relevant passages from digitized Islamic sources to refine model predictions.Our final system achieves an accuracy of 89.8% on the hidden test set.
Authors: Nourah Alangari, Nouf AlShenaifi
Institution: King Saud University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4416037624