The Pitfalls of AI-Assisted Social Science Research: A Case Study on Interpersonal Utility Measurement
Year: 2025
Type: article
Abstract: This study examines the challenges of using artificial intelligence (AI) in social science research, focusing on the author's experience with Grok 3 to investigate interpersonal utility measurement. Initially intended as a tool to update a prior publication on utilitarian ethics, the AI-generated citations were found to be largely fabricated, prompting a deeper exploration of AI's reliability. Through iterative interactions, the author uncovered systemic issues in AI's source validation, including fabricated references and incorrect bibliographic details. The study highlights the necessity of rigorous verification when using AI assistants, proposes transparent disclosure of potential inaccuracies, and suggests strategies for researchers to mitigate risks. This case study underscores broader implications for AI's role in academic research, emphasizing the need for critical oversight to ensure scholarly integrity.
Author: Robert W. McGee
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7083587385
FWCI:
Cites:
Cited by:
Related to: