Year: 2026
Type: conference-paper
Abstract: Low-resource agglutinative languages, characterized by rich morphological inflection and severe vocabulary sparsity in corpora, have long posed numerous challenges in the field of representation learning.Word-level representations preserve semantic integrity but struggle to handle sparse surface forms, whereas morpheme-level representations, though easier to learn, often lack holistic semantic information.Existing multi-granularity methods are typically modeled at the word and phrase levels, with very limited application to lowresource agglutinative languages.Focusing on the morphemes of agglutinative languages, this paper proposes MAGNet, a morphology-aware gated multi-granularity pre-training framework.At the morpheme granularity, this framework leverages morphological knowledge and integrates morpheme segmentation with morphological tagging to construct fine-grained representations.It further introduces a morphologyaware masked language modeling objective to facilitate the model in learning functional morphological regularities.Meanwhile, at the word granularity, a word-level encoder is employed to capture contextual semantics and maintain its semantic coherence.Finally, a gated fusion mechanism dynamically fuses representations of different granularities according to the context.Experiments conducted on two low-resource agglutinative languages, Mongolian and Turkish, for the tasks of dependency parsing and named entity recognition (NER) demonstrate that our method achieves consistent performance improvements over strong baseline models.Ablation studies further validate the complementary roles of morphological tagging and whole-word modeling in efficient representation learning.
Authors: Zhonghao Zhang, Na Liu, Jiajia Ma, Nier Wu, Guiping Liu
Institution: Inner Mongolia University of Technology
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166830572