Work

Word Segmentation of Informal Arabic with Domain Adaptation

Year: 2014

Type: conference-paper

Abstract: Segmentation of clitics has been shown to improve accuracy on a variety of Arabic NLP tasks.However, state-of-the-art Arabic word segmenters are either limited to formal Modern Standard Arabic, performing poorly on Arabic text featuring dialectal vocabulary and grammar, or rely on linguistic knowledge that is hand-tuned for each dialect.We extend an existing MSA segmenter with a simple domain adaptation technique and new features in order to segment informal and dialectal Arabic text.Experiments show that our system outperforms existing systems on broadcast news and Egyptian dialect, improving segmentation F 1 score on a recently released Egyptian Arabic corpus to 92.09%, compared to 91.60% for another segmenter designed specifically for Egyptian Arabic.

Source: Proceedings of the conference - Association for Computational Linguistics. Meeting

DOI: 10.3115/v1/p14-2034

Authors: Will Monroe, Spence Green, Christopher D. Manning

Institution: Stanford University

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: National Science Foundation, Defense Advanced Research Projects Agency, Advanced Research Projects Agency

OpenAlex ID: W2250659129