Year: 2025
Type: article
Abstract: The paper presents findings of the scientific and methodological investigation of word embedding models for application in natural language processing (NLP). The research is timely due to rapid advancements in computational power, enabling large-scale text analysis. The study is focused on adapting existing word embedding models — Word2Vec, GloVe, FastText, ELMo and BERT — to the under-represented Ukrainian language. While effective for English texts, these models are difficult to use for Ukrainian due to linguistic specifics and lack of quality resources. The objective is to compare these models for Ukrainian text processing and suggest improvements based on existing corpora. The paper reviews their architectures, training parameters, and highlights their pros and cons. FastText is effective with infrequent and morphologically rich words and is hence a good candidate for Ukrainian; Word2Vec is effective and simple to use; GloVe is effective for capturing global co-occurrence. Compared to these static embedding models, ELMo generates contextualized word representations that had better handle polysemy and syntactic variation, although it requires more computational resources. BERT further improves contextual understanding through its transformer-based bidirectional architecture, outperforming previous models in NLP tasks; however, it is more demanding in terms of memory and training time. Comparative assessment is given that illustrates how word frequency thresholds, context window size, and vector size affect accuracy in models. The study stresses relevence of tuning parameters to match task-specific and linguistic needs. The research also encompasses Ukrainian-language resources like Ukrainian Language Corpus (100M tokens), Brown Ukrainian Corpus (1M tokens), and community resources like lang.org.ua. However, there is a lack of available, annotated resources that limit larger-scale experiments. In conclusion, this work highlights the need for ongoing development of the corpus and better embedding model adaptation to Ukrainian. It makes recommendations to enhance accuracy and efficiency of models for educational, health care, legal, and mass media applications.
Authors: Dmytro Zhurko, Iryna Bilous
Institution: Chernihiv Polytechnic National University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W4410747239