Year: 2023
Type: conference-paper
Abstract: Despite the impressive growth of the abilities of multilingual language models, such as XLM-R and mT5, it has been shown that they still face difficulties when tackling typologically-distant languages, particularly in the low-resource setting.One obstacle for effective cross-lingual transfer is variability in word-order patterns.It can be potentially mitigated via source-or target-side word reordering, and numerous approaches to reordering have been proposed.However, they rely on language-specific rules, work on the level of POS tags, or only target the main clause, leaving subordinate clauses intact.To address these limitations, we present a new powerful reordering method, defined in terms of Universal Dependencies, that is able to learn fine-grained word-order patterns conditioned on the syntactic context from a small amount of annotated data and can be applied at all levels of the syntactic tree.We conduct experiments on a diverse set of tasks and show that our method consistently outperforms strong baselines over different language pairs and model architectures.This performance advantage holds true in both zero-shot and few-shot scenarios.1
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4389524237