Work

Transforming Causal LLM into MLM Encoder for Detecting Social Media Manipulation in Telegram

Year: 2025

Type: conference-paper

Abstract: We participated in the Fourth UNLP shared task on detecting social media manipulation in Ukrainian Telegram posts (Kyslyi et al., 2025), addressing both multilabel technique classification and token-level span identification.We propose two complementary solutions: for classification, we fine-tune the decoder-only model with class-balanced grid-search thresholding and ensembling.For span detection, we convert causal LLM into a bidirectional encoder via masked language modeling pretraining on large Ukrainian and Russian news corpora before fine-tuning.Our solutions achieve SOTA metric results on both shared task track.Our work demonstrates the efficacy of bidirectional pretraining for decoder-only LLMs and robust threshold optimization, contributing new methods for disinformation detection in low-resource languages.

DOI: 10.18653/v1/2025.unlp-1.13

Authors: Anton Bazdyrev, Ivan Bashtovyi, Ivan Havlytskyi, Oleksandr Kharytonov, Artur Khodakovskyi

Institution: National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

Language: English

Topic: Misinformation and Its Impacts

Subfield: Sociology and Political Science

Field: Social Sciences

Domain: Social Sciences

Open Access status: gold

Funder: Universidad Nacional de La Plata

OpenAlex ID: W4412944199

FWCI:

Cites:

Cited by:

Related to: 10