Noor-Ghateh: A Benchmark Dataset for Evaluating Arabic Word Segmentation Tools in the Hadith Domain
Year: 2020
Type: preprint
Abstract: Noor-Ghateh: A Benchmark Dataset for Evaluating Arabic Word Segmentation Tools in the Hadith Domain 📘 Overview Noor-Ghateh is a manually annotated Classical Arabic morphological dataset derived from the jurisprudential text Sharayeʿ al-Islam.The dataset provides fine-grained clitic segmentation, 14 morphological attributes, and gold-standard human annotation, making it a valuable benchmark for: Morphological analyzers Segmentation systems Lemmatizers & root extractors Classical Arabic NLP research Benchmarking domain sensitivity across analyzers The dataset includes 223,690 tokens, with a publicly available 313-token sample released in XML, JSON, and CSV-embedded-XML formats. 🧱 Data Format 1. XML Format (Primary) The XML structure uses → → hierarchy.Each element includes 14 morphological attributes such as: Seq — morpheme order Slice — surface form Affix — prefix/suffix/stem Pos, Lemma, Case, Categ, DervT, Num, Root TOV, Voic, Kol, Lang 2. JSON Format Direct JSON mapping of the XML hierarchy for machine learning pipelines. 3. CSV-embedded XML Each row contains:Surface form — Segmented form — XML annotation block 🎯 Intended Use Cases Training and evaluating morphological segmentation systems Testing classical Arabic analyzers (Farasa, CAMeL Tools, ALP, MADAMIRA) Building lemmatizers and root extractors Domain-sensitivity analysis Digital humanities research in Hadith & jurisprudence Linguistic studies of Classical Arabic morphology
Source: arXiv (Cornell University)
Authors: AlShuhayeb, Huda, Minaei-Bidgoli, Behrouz, Shenassa, Mohammad E., Hossayni, Sayyed-Ali
Institution: Iran University of Science and Technology
Language: Arabic
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4384918672