Advancing Arabic Diacritization: Improved Datasets, Benchmarking, and State-of-the-Art Models
Year: 2025
Type: conference-paper
Abstract: Arabic diacritics, similar to short vowels in English, provide phonetic and grammatical information but are typically omitted in written Arabic, leading to ambiguity.Diacritization (aka diacritic restoration or vowelization) is essential for natural language processing.This paper advances Arabic diacritization through the following contributions: first, we propose a methodology to analyze and refine a large diacritized corpus to improve training data quality.Second, we introduce WikiNews-2024, a multi-reference evaluation methodology with an updated version of the standard benchmark "WikiNews-2014".In addition, we explore various model architectures and propose a BiLSTM-based model that achieves state-of-the-art results with 3.12% and 2.70% WER on WikiNews-2014 and WikiNews-2024, respectively.Moreover, we develop a model that preserves user-provided diacritics while maintaining accuracy.Lastly, we demonstrate that augmenting training data enhances performance in low-resource settings.
Authors: Abubakr Mohamed, Hamdy Mubarak
Institution: Hamad bin Khalifa University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4416036175