Year: 2026
Type: conference-paper
Abstract: We present MeiteiRoBERTa, the first publicly available monolingual RoBERTa-based language model for Meitei (Manipuri), a lowresource language spoken by over 1.8 million people in Northeast India.Trained from scratch on 76 million words of Meitei text in Bengali script, our model achieves a perplexity of 65.89, representing a 5.2× improvement over multilingual baselines mBERT (341.56) and MuRIL (355.65).We argue that Meitei's agglutinative morphology and complex word formation present typological challenges that multilingual models with broad language coverage fail to capture effectively.Through comprehensive evaluation on perplexity, tokenization efficiency, and semantic representation quality, we demonstrate that domain-specific pretraining significantly outperforms general-purpose multilingual models for low-resource languages.Our model exhibits superior semantic understanding with 0.769 similarity separation compared to 0.035 for mBERT and near-zero for MuRIL, despite MuRIL's better tokenization efficiency (fertility: 3.29 vs. 4.65).We publicly release the model, training code, and datasets to accelerate NLP research for Meitei and other underrepresented Northeast Indian languages.
Author: Badal Nyalang
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W7140101172
FWCI:
Cites:
Cited by:
Related to: