Usability evaluation of spoken humanoid embodied conversational agents in mobile serious games
Year: 2019
Type: dissertation
Abstract: The use of embodied conversational agents (ECAs) and spoken dialogue systems in serious games offers theoretical advantages such as a more natural interaction with an agent displaying characteristics like personality, engagement, enjoyment, trust and emotions. Despite these theoretical advantages, according to recent studies, the interaction with spoken dialogue systems, either in the form of an embodied agent or not, is still inferior compared to other approaches that allow a direct manipulation of the system. However, the way users interact with mobile devices is rapidly changing, since the latest generation of mobile devices include voice driven virtual assistants (Apple Siri, Google Now, Samsung S Voice, Amazon Alexa). Previous research has focused on the design aspects of ECAs but there are limited empirical evaluations regarding their effectiveness in serious games and mobile serious games. In an era where usability has become an integral part of the development process, introducing ECAs in these environments without proper evaluation can be problematic. Thus, there is a strong reason to examine if ECAs enhance usability over current interaction paradigms in serious game environments, even more so in mobile devices as there is a recent trend towards mobile serious games. The research presented here investigates, across a series of two large scale experiments and a survey, the extent to which spoken Humanoid Embodied Conversational Agents (HECAs) can foster usability in mobile serious game (MSG) applications. The aim of the research is to assess the impact of multiple agents, serious game approaches and illusion of humanness on the quality of the interaction. The first experiment (pilot study 1) investigates whether the portrayal of an application as a game (with game-like implicit feedback) influences the overall usability of a virtual application. The main purpose of this study is to act as a methodological sandbox to inform the methodology approaches of the main experiment. Qualitative analysis of the experiment shows that 78% of participants prefer the game version. Also, the game version was perceived as more fun, enjoyable and stimulating. Results of the survey (study 2) show that 83% of the participants have played games within the last 6 months and 60% of them play games on their smartphone even though they also own laptops as well as desktops and game consoles. The device of choice for everyday activities is also the smartphone. Moreover, most participants replied that they owned a smartphone with a screen size over 5”. The data collected from the preliminary studies informed the hardware, methodological and design decisions of the main experiment. The main mobile experiment investigates two styles of agent presentation, an agent of high human-likeness (HECA) and an agent of low human-likeness (text). The purpose of the experiment is to access how agents of high human-likeness can evoke the illusion of humanness and affect usability. Agents of high human-likeness were designed by following the ECA design model that is a proposed guide for ECA development. The results of the experiment show that users prefer to interact with the HECAs. The difference between the two versions is statistically significant with a large effect size and many of the participants justifying their choice by saying that the human-like characteristics of the HECA made the version more appealing. This research provides key information on the potential effect of HECAs on serious games, which will likely impact the design decisions regarding spoken HECAs and the design of future mobile serious games.
Source: ERA
Author: Danai Korre
Language: English
Subfield: Social Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W3001411746