Cognitive Biases in Large Language Model based Decision Making: Insights and Mitigation Strategies
Year: 2025
Type: article
Abstract: Recent development of Artificial Intelligence (AI) has led to an increasing integration of Large Language Models (LLMs) for real-world decision-making. While these models can enhance decision-making processes, they remain susceptible to the same cognitive biases found in humans. Our study examines two well-documented biases found in LLMs–Anchoring and Framing effect–and evaluates their presence in OpenAI’s GPT-3.5 turbo, GPT-4o, and GPT-4o-mini. Using a structured framework, we assess the degree of these biases across different models and proposed four bias mitigation prompting strategies: Chain-of-Thought (CoT), Thoughts of Principles (ToP), ignoring anchor hints, and reflection. We evaluated the bias reduction effectiveness of each prompting strategy using both our proposed prompts and GPT-generated prompts. Our findings reveal significant differences in bias levels across models (p < 0.01), with stronger models exhibiting a greater susceptibility to bias. However, none of the tested prompting strategies indicated statistically significant effectiveness in bias reduction. Our results highlight the challenges of debiasing LLMs and underscore the need for caution when deploying them in real-world decision-making scenarios. Further research is necessary to explore broader cognitive biases and develop more effective mitigation strategies.
Author: Siduo Chen
Institution: Emory University
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: hybrid
OpenAlex ID: W4408421332