Work

Towards Simulating Social Media Users with LLMs: Evaluating the Operational Validity of Conditioned Comment Prediction

Year: 2026

Type: conference-paper

Abstract: The transition of Large Language Models (LLMs) from exploratory tools to active "silicon subjects" in social science lacks extensive validation of operational validity.This study introduces Conditioned Comment Prediction (CCP), a task in which a model predicts how a user would comment on a given stimulus by comparing generated outputs with authentic digital traces.This framework enables a rigorous evaluation of current LLM capabilities with respect to the simulation of social media user behavior.We evaluated open-weight 8B models (Llama3.1,Qwen3, Ministral) in English, German, and Luxembourgish language scenarios.By systematically comparing prompting strategies (explicit vs. implicit) and the impact of Supervised Fine-Tuning (SFT), we identify a critical form vs. content decoupling in lowresource settings: while SFT aligns the surface structure of the text output (length and syntax), it degrades semantic grounding.Furthermore, we demonstrate that explicit conditioning (generated biographies) becomes redundant under fine-tuning, as models successfully perform latent inference directly from behavioral histories.Our findings challenge current "naive prompting" paradigms and offer operational guidelines prioritizing authentic behavioral traces over descriptive personas for high-fidelity simulation.

Source: Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA)

DOI: 10.18653/v1/2026.wassa-1.16

Authors: Nils Schwager, Simon Münker, Alistair Plum, Achim Rettinger

Institutions: Universität Trier, University of Luxembourg

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: HORIZON EUROPE Framework Programme

OpenAlex ID: W7140079217