A hybrid uncertainty-aware deep learning framework for reliable Malayalam fake news detection
Year: 2026
Type: article
Abstract: Fake news detection in low resource languages like Malayalam is a challenging task due to linguistic complexity, limited linguistic resources and lack of annotated datasets. The deep learning models has proven good impact in text classification tasks, but they lack reliability and interpretability that are critical for real world deployment. There are some recent studies that focus on explainable artificial intelligence to interpret model predictions but the reliability of these explanations under uncertain predictions has not been addressed. This paper proposes an uncertainty aware framework for Malayalam fake news detection using different deep learning architectures, including Bidirectional LSTM (BiLSTM), CNN-LSTM, Bidirectional GRU (BiGRU), CNN-GRU, and CNN-BiLSTM. Four uncertainty estimation techniques Monte Carlo (MC) Dropout, Deep Ensemble Learning, Test Time Augmentation (TTA) and Evidential Deep Learning (EDL) are used and compared with baseline models. Along with fake news classification, the proposed framework quantifies predictive uncertainty through entropy, Kullback-Leibler (KL) divergence, mutual information, variance, and vacuity measures, enabling the identification of unreliable predictions. A hybrid uncertainty framework combining MC Dropout, Deep Ensemble Learning and Evidential Learning is proposed to jointly capture epistemic and aleatoric uncertainty while providing a complementary evidence based uncertainty estimates. The proposed hybrid framework achieves an accuracy of 96.51%, precision of 96.52%, recall of 96.51%, and F1-score of 96.51%. To assess the robustness of proposed hybrid model result and address variability due to random initialization, an additional five-seed experiment was conducted. The Hybrid model achieved an accuracy of 95.85 ± 0.18, precision of 95.85 ± 0.18%, recall of 95.85 ± 0.18%, and F1-score of 95.85 ± 0.18%, indicating stable performance across multiple runs. Furthermore, the calibration analysis demonstrates substantial improvements in prediction reliability, achieving the lowest Expected Calibration Error (ECE) of 0.0066, Negative Log-Likelihood (NLL) of 0.1014, and Brier Score of 0.0546 among all evaluated models. The five seed experiment achieved an ECE of ( $$0.0067 \pm 0.0023$$ ), MCE ( $$0.0955 \pm 0.0494$$ ), NLL ( $$0.1229 \pm 0.0027$$ ), and Brier score ( $$0.0659 \pm 0.0007$$ ) for the proposed hybrid model, which is lower than all other models. Although the Deep Ensemble and MC Dropout + Ensemble models achieve slightly higher mean classification accuracy, the proposed Hybrid Model achieves the lowest ECE, MCE, NLL, and Brier score across the five-seed experiments, indicating improved calibration and probabilistic prediction quality. Additional evaluation using uncertainty specific metrics such as Uncertainty Accuracy Coefficient (UAC), Uncertainty Precision Ratio (UPR), Uncertainty Sensitivity Estimate (USE), and Uncertainty Specificity Percentage (USP) confirms the effectiveness of the proposed framework in identifying uncertain predictions while maintaining high classification performance. The experimental evaluations demonstrates that uncertainty aware models provide better calibration and improved reliability compared to baseline architectures while maintaining competitive classification performance. The results shows that the hybrid uncertainty modeling effectively reduces overconfidence and enhances trustworthiness by distinguishing between confident and uncertain samples, that highlights the importance of uncertainty quantification in low resource language fake news detection.
Source: Scientific Reports
Institutions: Amrita Vishwa Vidyapeetham, Universitat Pompeu Fabra
Language: English
Subfield: Sociology and Political Science
Field: Social Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $2,690
OpenAlex ID: W7225324869