Work

Ara-HOPE: Human-Centric Post-Editing Evaluation for Dialectal Arabic to Modern Standard Arabic Translation

Year: 2026

Type: other

Abstract: This paper addresses key gaps in assessing Dialectal Arabic to Modern Standard Arabic (DA-MSA) translation quality, where standard automatic metrics and general-purpose human evaluation methods fail to capture dialect-specific errors. We introduce Ara-HOPE, a human-centric post-editing evaluation framework comprising a five-category error taxonomy and a decision-tree annotation protocol designed for consistent use by minimally trained native dialect speakers. Using Ara-HOPE, we comparatively evaluate three mainstream systems (Jais, GPT-3.5, and NLLB-200) to quantify error patterns, pinpoint systematic weaknesses, and provide actionable guidance for improving dialect-aware MT. Our analysis shows clear performance differences across systems and highlights dialect-specific terminology handling and semantic preservation as the most persistent challenges in DA-MSA translation.

Source: Underline Science Inc.

DOI: 10.48448/zf2t-bz96

Authors: Association for Computational Linguistics 2026, Alabdullah, Abdullah, Han, Lifeng, Lin, Chenghua

Institutions: University of Edinburgh, Leiden University, University of Manchester

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W7155388965

Cites:

Cited by:

Related to: