Comparing Methods for Multi-Label Classification of Manipulation Techniques in Ukrainian Telegram Content
Year: 2025
Type: conference-paper
Abstract: Detecting manipulation techniques in online text is vital for combating misinformation, a task complicated by generative AI.This paper compares machine learning approaches for multi-label classification of 10 techniques in Ukrainian Telegram content (UNLP 2025 Shared Task 1).Our evaluation included TF-IDF, fine-tuned XLM-RoBERTa-Large, PEFT-LLM (Gemma, Mistral) and a RAG approach (E5 + Mistral Nemo).The fine-tuned XLM-RoBERTa-Large model, which incorporates weighted loss to address class imbalance, yielded the highest Macro F1 score (0.4346).This result surpassed the performance of TF-IDF (Macro F1 0.32-0.36), the PEFT-LLM (0.28-0.33) and RAG (0.309).Synthetic data slightly helped TF-IDF but reduced transformer model performance.The results demonstrate the strong performance of standard transformers like XLM-R when appropriately configured for this classification task.
Author: Oleh Melnychuk
Institution: National University "Kyiv Aviation Institute"
Language: English
Subfield: Sociology and Political Science
Field: Social Sciences
Domain: Social Sciences
Open Access status: gold
OpenAlex ID: W4412944117