Work

Improving the Time Efficiency of a Script Identification Algorithm Using a Unicode-Based Regular Expression Matching Strategy

Year: 2026

Type: article

Abstract: Script identification is the first step in most multilingual text-processing systems. To improve the time efficiency of script identification algorithms, whether there is content written in a certain script in the text is first determined; if so, the content written in that script is then obtained. Then, it is determined whether the total length of the texts corresponding to the identified scripts is equal to the original text length; if so, the script identification process ends. Finally, considering the frequencies of various scripts on the Internet, those that are more common are prioritized during script identification. Based on these three approaches, an improved script identification algorithm was designed. A comparison experiment was conducted using sentence-level text corpora in 263 languages written in 26 scripts. The testing times of the newly proposed method were reduced by 9.35-fold, while the F1 score for script identification was slightly higher than those reported in our earlier studies. The method proposed in this study effectively improves the time efficiency of script identification algorithms.

Source: Applied Sciences

DOI: 10.3390/app16041714

Authors: Mamtimin Qasim, Wushour Silamu, Yasen Aizezi

Institutions: Nanfang College Guangzhou, Xinjiang University, Xinjiang Police College

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,896

Funders: National Natural Science Foundation of China, Natural Science Foundation of Xinjiang