Work

Generative AI responses are a dime a dozen; Making them count is the challenge – Evaluating information presentation styles in healthcare chatbots using hierarchical Bayesian regression models

Year: 2025

Type: article

Abstract: The emergence of large language models offers new opportunities to deliver effective healthcare information through web-based healthcare chatbots. Health information is often complex and technical, making it crucial to design human-AI interactions that effectively meet user needs. Employing a 2x2 between subjects design, we controlled for two independent variables: communication style (conversational vs. informative) and language style (technical vs. non-technical). We used hierarchical Bayesian regression models to assess the impact varying information presentation styles on effectiveness, trustworthiness, and usability. The findings revealed perceptions of low usability significantly decreased the effectiveness of the healthcare chatbot. Additionally, participants exposed to the conversational style of the chatbot had significantly increased likelihoods to perceive it with higher usability but were also more likely to be less trusting of the chatbot. These results indicate varying information presentation styles can impact user experience and offers insights for future research with healthcare chatbots and other AI systems.

Source: Applied Ergonomics

DOI: 10.1016/j.apergo.2025.104515

Authors: Samuel N. Koscelny, Sara Sadralashrafi, David M. Neyens

Institution: Clemson University

Language: English

Topic: Artificial Intelligence in Healthcare and Education

Subfield: Health Informatics

Field: Medicine

Domain: Health Sciences

SDG: Good health and well-being

Open Access status: hybrid

Estimated APC paid: $3,400