Empowering Human-AI Collaboration with AR: The Importance of Visual Illustration and Verbalized Instruction
Year: 2025
Type: article
Abstract: This study investigates how verbal and visual instructional formats influence human-AI collaboration in augmented reality (AR) environments designed for technical diagnosis. Drawing on dual-coding theory, the authors distinguish between inductive and deductive instructions and schematic and structural illustrations, proposing that cognitive alignment between these modalities, termed processing consistency, enhances performance. Two experiments revealed that inductive instructions improve diagnostic efficiency with schematic visuals but reduce it with structural visuals. Conversely, deductive instructions paired with structural visuals enhance the effectiveness. The study also examined how cognitive effort and confused emotions mediate these effects, finding that humanoid verbalizations engage both mechanisms, while robotic voices primarily evoke emotional confusion. These findings offer theoretical insights into multimodal information processing and practical guidance for designing effective AR-based AI systems for collaborative tasks, particularly in contexts requiring critical decision-making and spatial reasoning.
Authors: Hong, Zeyuan (Stephen), Choi, Ben, Boh, Waifong
Institution: Nanyang Technological University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: green
OpenAlex ID: W7028409355
FWCI:
Cites:
Cited by:
Related to: