Work

YallaMorph: A Benchmark for Evaluating Arabic Morphological Generation in Large Language Models

Year: 2026

Type: preprint

Abstract: Arabic morphology remains challenging for large language models, since fluent generation does not guarantee accurate morphosyntactic control. Existing Arabic evaluations mainly target downstream tasks and do not directly test controlled morphological generation from explicit lexical and feature-based input. We introduce YallaMorph, a large-scale benchmark for Arabic morphological generation covering verbs, nouns, adjectives, their cliticized forms, and invalid configurations. We evaluate multilingual and Arabic-oriented LLMs under diacritized and undiacritized settings over 600K benchmark entries. Results show that Arabic morphological generation remains difficult, especially for cliticized, unseen, and morphologically rare forms.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2609.10153

Authors: Reda, Mahmoud, Khalifa, Salam, Marzouk, Reham, Habash, Nizar

Institutions: New York University Abu Dhabi, Stony Brook University, Mohamed bin Zayed University of Artificial Intelligence

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: New York University Abu Dhabi, York University

OpenAlex ID: W7212150586

Cites:

Cited by:

Related to: