Year: 2025
Type: article
Abstract: Understanding how large language models perform relative to humans in socially interactive, deduction-based tasks is vital for advancing AI applications. This study compares the performance of human players and GPT-4o in Guess vs. AI, a custom strategic deduction game. Drawing on data from 85 completed games, the AI-opponent achieved a significantly higher win rate than human players (63.5%, p = 0.009) and required fewer questions to identify the target (humans: 17, AI-opponent: 9). These findings highlight GPT-4o’s strengths in systematic reasoning, pattern recognition and efficient decision-making. While showcasing the potential of large language models in structured deduction scenarios, they also emphasize the need for further research into AI adaptability in more socially complex tasks. Future directions include expanding demographic diversity, exploring additional game formats or different large language models and investigating potential human-AI collaborations rather than strictly competitive environments.
Authors: Tinke, Pjotr, von Mentlen, Thomas
Institution: KTH Royal Institute of Technology
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6982330197
FWCI:
Cites:
Cited by:
Related to: