Work

Applying Usability Testing to Conversational Systems in an Internet of Things Context: An Industry Experience Report

Year: 2026
Type: conference-paper
Abstract: The use of conversational systems has intensified in real-world applications, particularly in Internet of Things scenarios, where natural language interaction is considered a means of reducing system complexity. However, usability and user experience issues still affect the adoption and effectiveness of such systems. This paper presents an experience report on the conduct of usability tests on a conversational system applied in an Internet of Things context, aiming to analyze user experience and identify usability problems from the perspective of end users. The experience was structured using the Goal–Question–Metric framework, which supported the definition of analysis goals, guiding questions, and evaluation metrics. The usability tests involved representative tasks and the collection of quantitative and qualitative data, including task success rate, execution time, total questions per task, and users’ subjective perceptions. The results highlight recurring issues related to understanding system responses, feedback during interaction, and the adequacy of the conversational flow for the usage context. The findings provide exploratory insights into the design and evaluation of conversational systems, focusing on human and contextual aspects in human–computer interaction.
Language: English
Open Access status: diamond
OpenAlex ID: W7204256643
FWCI:
Cites: 10
Cited by:
Related to: 10