Blending Queries and Conversations: Understanding Trust, Verification, and System Choice in Search and Chat Interactions
Year: 2025
Type: conference-paper
Abstract: This paper presents a user study (N=22) where participants used an interface combining Web Search and a Generative AI-Chat feature to solve health-related information tasks.We study how people behaved with the interface, why they behaved in certain ways, and what the outcomes of these behaviours were.A think-aloud protocol captured their thought processes during searches.Our findings suggest that GenAI is neither a search panacea nor a major regression compared to standard Web Search interfaces.Qualitative and quantitative analyses identified 78 tactics across five categories and provided insight into how and why different interface features were used.We find evidence that pre-task confidence and trust both influenced which interface feature was used.In both systems, but particularly when using the chat feature, trust was often misplaced in favour of ease-of-use and seemingly perfect answers, leading to increased confidence post-search despite having incorrect results.We discuss what our findings mean in the context of our defined research questions and outline several open questions for future research.
Authors: Kerstin Mayerhofer, Rob Capra, David Elsweiler
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4409973548