Work

Assessing the Linguistic Knowledge in Arabic Pre-trained Language Models Using Minimal Pairs

Year: 2022

Type: conference-paper

Abstract: Despite the noticeable progress that we recently witnessed in Arabic pre-trained language models (PLMs), the linguistic knowledge captured by these models remains unclear.In this paper, we conducted a study to evaluate available Arabic PLMs in terms of their linguistic knowledge.BERT-based language models (LMs) are evaluated using Minimum Pairs (MP), where each pair represents a grammatical sentence and its contradictory counterpart.MPs isolate specific linguistic knowledge to test the model's sensitivity in understanding a specific linguistic phenomenon.We cover nine major Arabic phenomena from: Verbal sentences, Nominal sentences, Adjective Modification, and Idafa construction.The experiments compared the results of fifteen Arabic BERT-based PLMs.Overall, among all tested models, CAMeL-CA and GigaBERT outperformed the other PLMs by achieving the highest overall accuracy.

Source: Arabic Natural Language Processing Workshop (WANLP)

DOI: 10.18653/v1/2022.wanlp-1.17

Authors: Wafa Abdullah Alrajhi, Hend Al-Khalifa, Abdulmalik AlSalman

Institution: King Saud University

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

OpenAlex ID: W4385573904

FWCI: 0.138

Cites: 18

Cited by: 1

Related to: 10