Work

Fine-tuned large language models can replicate expert coding better than trained coders: a study on informative signals sent by interest groups

Year: 2026

Type: article

Abstract: Understanding how political information is transmitted requires tools that can reliably and scalably capture complex signals in text. While existing studies highlight interest groups as strategic information providers, empirical analysis has been constrained by reliance on expert annotation. Using policy documents released by interest groups, this study shows that fine-tuned large language models (LLMs) outperform lightly trained workers, crowdworkers, and zero-shot LLMs in distinguishing two difficult-to-separate categories: informative signals that help improve political decision-making and associative signals that shape preferences but lack substantive relevance. We further demonstrate that the classifier generalizes out of distribution across two applications. Although the empirical setting is domain-specific, the approach offers a scalable method for expert-driven text coding applicable to other areas of political inquiry.

Source: Political Science Research and Methods

DOI: 10.1017/psrm.2025.10086

Authors: Dahyun Choi, Denis Peskoff, Brandon M. Stewart

Institution: Princeton University

Language: English

Topic: Computational and Text Analysis Methods

Subfield: General Social Sciences

Field: Social Sciences

Domain: Social Sciences

SDG: Peace, justice, and strong institutions

Open Access status: gold

Estimated APC paid: $3,550