Character, Word and Dual-Granularity TF-IDF Representations for Stance Detection in Low-Resource Manipuri (Meetei Mayek) Editorials: Near-Duplicate Leakage Reverses the Ranking
Year: 2026
Type: article
Abstract: Document-level stance detection has received little attention for low-resource languages written in scripts with little digital text. We compare word-level, character-level and concatenated word-plus-character (Dual-Granularity Lexical Fusion, DGLF) TF-IDF representations for three-class stance detection on 1575 Manipuri editorials in Meetei Mayek script. Auditing the cross-validation grouping inherited from earlier work, we find that it missed most near-duplicate reprints: grouping by character-5-gram similarity places 385 documents (24.4%) in near-duplicate groups rather than 104, including 282 of the 336 FOR editorials. Under near-duplicate-safe grouping, with every method run on the same eight seeds and folds, the scores of all three representations fall, by 0.08 to 0.17 macro-F1, and their ranking changes. Character-only features move from last to first (document-level macro-F1: 0.590), DGLF improves on word-only features (0.563 vs. 0.520; permutation p < 0.001) but falls below character-only features, clearly with a linear SVM (Holm-adjusted p = 0.004) and suggestively with logistic regression (p = 0.024, Holm-adjusted: 0.084), and elastic-net regularisation, a convolutional network and frozen contextual-embedding fusion do not surpass character-only features. Under the original grouping, word n-grams had benefited from memorising reprinted editorials. We recommend similarity-based near-duplicate grouping for small newspaper corpora and offer character n-gram TF-IDF as the baseline to beat.
Source: Electronics
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,896
OpenAlex ID: W7222976397