Year: 2026
Type: conference-paper
Abstract: Language identification (LID) is crucial for natural language processing systems, yet Northeast Indian languages remain severely underserved by existing multilingual LID models.We present NE-LID, a fast and accurate language identification system specifically designed for eleven languages of Northeast India.Built using character n-gram features with fastText, NE-LID achieves 99.09% accuracy on a balanced test set, significantly outperforming existing multilingual systems including GlotLID (73.12%),OpenLID (42.03%),IndicLID (39.30%), and LangDetect (24.33%).Our model processes predictions in 0.084 milliseconds on average, enabling real-time applications.We demonstrate that character-level modeling outperforms transformer-based approaches for script-diverse, low-resource languages.
Author: Badal Nyalang
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166552453