Year: 2025
Type: conference-paper
Abstract: In this paper, we report our participation to the PalmX cultural evaluation shared task.Our system, CultranAI, focused on data augmentation and LoRA fine-tuning of large language models (LLMs) for Arabic cultural knowledge representation.We benchmarked several LLMs to identify the best-performing model for the task.In addition to utilizing the PalmX dataset, we augmented it by incorporating the Palm dataset and curated a new dataset of over 22K culturally grounded multiple-choice questions (MCQs).Our experiments showed that the Fanar-1-9B-Instruct model achieved the highest performance.We fine-tuned this model on the combined augmented dataset of 22K+ MCQs.On the blind test set, our submitted system ranked 5th with an accuracy of 70.50%, while on the PalmX development set, it achieved an accuracy of 84.1%.We made experimental scripts publicly available for the community.1
Institutions: Qatar University, University of Toronto
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4416037784