ISL-NLP at PalmX 2025: Retrieval-Augmented Fine-Tuning for Arabic Cultural Question Answering
Year: 2025
Type: conference-paper
Abstract: Cultural understanding is essential for large language models (LLMs), particularly in the Arabic context where many models struggle to capture nuanced cultural elements.To address this gap, we propose a novel approach for Arabic cultural multiple-choice question answering that integrates retrieval-based training data augmentation with parameter-efficient fine-tuning.Our system employs Gemini 1 to retrieve contextual evidence for each question, selects candidate pairs, and adapts NileChat-3B by fine-tuning only three projection layers, reducing trainable parameters by 68.2% while preserving general language proficiency.On the PalmX 2025 Subtask 1 benchmark 2 , our system attains 67.60% accuracy on the blind test set, ranking 6 th overall and outperforming the NileChat-3B baseline by 3% on the development set.The model weights are publicly available at MohamedGomaa30/Ibn-Al-Nafs.
Authors: Mohamed Gomaa, Noureldin Elmadany
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4416037609