Year: 2026
Type: conference-paper
Abstract: Multilingual transformers have achieved remarkable performance on code-mixed sentiment benchmarks, but their robustness under linguistic stress and domain shift remains underexplored.We fine-tune XLM-RoBERTa and mBERT on a carefully cleaned 25,543tweet Hinglish sentiment dataset, where XLM-R achieves near-perfect in-distribution accuracy (99.7%).The integrity of this result is confirmed by rigorous hash-based and 3-gram Jaccard deduplication, ruling out data leakage.However, when evaluated on a 400-example human-validated adversarial benchmark spanning negation, sarcasm, contrast, subtle sentiment, and true neutral, XLM-R performance collapses to 42.5% -a drop of over 57 percentage points.Zero-shot transfer to English TweetEval yields only 50.8% accuracy (40.8% macro F1), above .Our results highlight a critical gap between benchmark scores and realworld reliability, underscoring the need for adversarial evaluation and cross-domain stresstesting before deploying sentiment models in practical, safety-sensitive applications.
Authors: Chennuru Rahul, Kolawole Adebayo
Institution: National University of Ireland, Maynooth
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166880533