Work

Large-Scale User Behavior Analysis in Multimodal AI-Assisted Manual Task Execution

Year: 2026

Type: preprint

Abstract: Conversational Task Assistants (CTAs) are multimodal dialogue systems that support users in complex real-world tasks such as cooking and DIY through voice, text, image, and video interactions. Prior user studies have focused on controlled settings, leaving limited understanding of real-world CTA usage at scale. In this work, we present a large-scale study of CTA usage based on thousands of users in-the-wild. Our large-scale real-world data analysis unveils new understandings of (i) user-CTA interaction flows, (ii) user intents, (iii) user conversational traits, and (iv) behavioral factors associated with user satisfaction. Our findings reveal key opportunities for future research in CTAs, particularly in user interaction design and task engagement, concluding with concrete design guidelines.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2609.07594

Authors: Ferreira, Rafael, Tavares, Diogo, Glória-Silva, Diogo, Semedo, David, Magalhães, João

Institutions: Universidade Nova de Lisboa, Laboratório para a Ciência da Computação e Informática

Topic: Speech and dialogue systems

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: NOVA Laboratory for Computer Science and Informatics, Fundação para a Ciência e a Tecnologia

Awards: NOVA Laboratory for Computer Science and Informatics, Métodos para manutenção de estado e gestão de políticas de diálogos em agentes de conversação, G5091561280

OpenAlex ID: W7212003678

Cites:

Cited by:

Related to: