Year: 2026
Type: dissertation
Abstract: As machine learning systems—particularly language models—become increasingly central to working with data and modern text processing, their applicability to historical and pre-internet texts remains underexplored. Most models are trained on contemporary internet-era corpora, raising questions about their performance on older linguistic data and their ability to capture semantic drift and diachronic change. This study investigates the diachronic robustness of BERT-based architectures across English and Russian, evaluating how well modern models handle language variation over time. Through comparative analysis of monolingual and multilingual BERT variants, we examine accuracy decay across centuries, assess the impact of orthographic reform and lexical shift, attempt to capture semantic shift and explore strategies for improving historical text understanding. Findings highlight significant performance drops of prediction of randomly masked tokens on archaic texts, variable by language, and demonstrate that multilingual pre-training and using larger models offer viable paths toward diachronic resilience.
Author: Maksimov, Evgenii
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7161211279