Enhancing Human-AI Collaboration in Decision Making: From Computational Behavior Insights to Intelligent Interventions Design
Year: 2026
Type: dissertation
Abstract: As artificial intelligence (AI) becomes increasingly integrated into human problem-solving, human–AI collaboration for decision making has emerged in a variety of forms, differing in how responsibility and task-solving roles are delegated between humans and AI systems. In one prevalent mode, often referred to as AI-assisted decision making, the AI model provides recommendations while human decision makers retain responsibility for the final decision. In many real-world scenarios that are not highly critical, however, humans need not be involved in every individual decision, which allows AI systems to take a more active role in solving tasks while humans step back to guide and supervise the overall process. Such a shift in role allocation gives rise to a second mode, often referred to as human-in-the-loop AI systems. Regardless of which collaboration model is adopted, effective human–AI collaboration requires careful design of how humans and AI systems interact. Without a thoughtfully designed collaboration process, humans often exhibit suboptimal behavior, either over-relying on incorrect AI outputs, wrongly rejecting accurate ones, or failing to effectively guide AI systems toward desired outcomes. Existing efforts to address these challenges typically rely on static, heuristic-based intervention strategies. Unfortunately, these previous approaches often operate under an isolated view of human cognition, where once users are given the “right” collaboration strategy, they are expected to interpret and behave as designers predict based on heuristic experience. In reality, human behavior and cognition, shaped by individual preferences and contextual factors, are dynamic and rarely conform to such expectations, leading to the frequent failure of these strategies. This dissertation argues that effective human-AI collaboration requires the computational design of interaction mechanisms that are informed by human behavior and cognition. I begin with the AI-assisted decision making paradigm, where AI explanations are increasingly adopted to help humans better understand and appropriately rely on AI recommendations. Building on a quantitative model of how humans integrate AI recommendations and explanations into their decisions, I show that explanations strategically tailored for a benign objective can significantly improve team performance even while the underlying AI model remains unchanged. When AI explanations are unavailable, I further consider how large language models (LLMs) can serve as auxiliary task analyzers. I propose a framework that adaptively presents LLM-powered analyses to decision makers based on a quantitative model of how humans react to different analyses, and human-subject evaluations show that it improves humans’ appropriate reliance on AI models. When no complementary information is available, I shift from adaptive information augmentation to adaptive interaction paradigm design. I propose a human-adaptive interaction framework that uses a contextualized multi-armed bandit algorithm to leverage feedback from individual decision makers and dynamically identify the optimal form of AI assistance for the current task. Finally, I turn to the human-in-the-loop paradigm, where the role of interaction design is no longer to shape how humans receive AI assistance but how AI receives human guidance. Taking feature engineering for tabular learning as the task, I propose a human–LLM collaborative framework that selectively elicits and incorporates human expert preference feedback into the feature engineering process, improving performance across diverse tabular datasets while reducing users’ cognitive load. In conclusion, this dissertation presents a unified view of human–AI collaboration through computationally designed, human-aware interaction mechanisms that span both AI-assisted decision making and human-in-the-loop AI systems, moving beyond fixed heuristic collaboration strategies toward adaptive and intelligent interaction design.
Source: Purdue
Author: Zhuoyan Li
Institution: Purdue University System
Language: English
Subfield: Social Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7208770201