Year: 2026
Type: conference-paper
Abstract: This project addresses the general absence of Natural Language Processing (NLP) tools when it comes to historical languages as a subset of low-resource languages that is relevant to an array of academic disciplines from linguistics to textual criticism.In particular, we train an Ancient Hebrew language model, BEReshiT, as well as BEReshiT-morph, a submodel for morphological annotation.BEReshiT is achieved through the fine-tuning of DictaBERT, a state-of-the-art model for Modern Hebrew that has also proved useful in Biblical Hebrew tasks.Layer freezing is applied in order to achieve maximal results while gaining insight about the adaptation process.In the context of an elaborate cloze test, BEReshiT demonstrates increased performance and notions of the Ancient Hebrew language compared to the source model as well as a selection of additional relevant models.The submodel BEReshiTmorph performs highly on tasks of morphological classification, reaching an F1 score of 0.97 for part-of-speech (POS) tagging.We release the main and morphological models as well as the datasets used at training and evaluation.
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166570842
FWCI:
Cites:
Cited by:
Related to: