Year: 2025
Type: article
Abstract: The advent of generative AI platforms and large language models (LLMs) such as ChatGPT has prompted scholarly work in two seemingly disconnected directions: automated classification as well as bias detection. Here these two strands of work are brought together to take up one of the larger challenges facing social scientific research with AI platforms: the effects of LLM safety guardrails on the quality of LLM data labelling. The piece briefly reviews the literature that takes up classification and bias, particularly their conjunction, which has been termed the safety/helpfulness trade-off. We then turn to findings made from research that explores the effects of guardrails on labelling. In all we find that the greater the bias mitigation the more neutralising sentiment exhibited by LLMs in their classification and labelling. By way of conclusion, we discuss the implications of this bias towards neutrality as an analytical flattening that accompanies the automation of knowledge making.
Source: Communication and Change
Authors: Richard Rogers, Xiaoke Zhang
Institutions: University of Amsterdam, Renmin University of China
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: hybrid
Estimated APC paid:
OpenAlex ID: W4414518797