Allen Institute for AI @ SIGTYP 2024 Shared Task on Word Embedding Evaluation for Ancient and Historical Languages
Year: 2024
Type: conference-paper
Abstract: In this paper, we describe Allen AI's submission to the constrained track of the SIG-TYP 2024 Shared Task.Using only the data provided by the organizers, we pretrained a transformer-based multilingual model, then finetuned it on the Universal Dependencies (UD) annotations of a given language for a downstream task.Our systems achieved decent performance on the test set, beating the baseline in most language-task pairs, yet struggles with subtoken tags in multiword expressions as seen in Coptic and Ancient Hebrew.On the validation set, we obtained ≥70% F1score on most language-task pairs.In addition, we also explored the cross-lingual capability of our trained models.This paper highlights our pretraining and finetuning process, and our findings from our internal evaluations.
Author: Lester James V. Miranda
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W7126368208
FWCI:
Cites:
Cited by:
Related to: