Enhancing Visitor Engagement in Interactive Art Exhibitions with Visual-Enhanced Conversational Agents
Year: 2025
Type: conference-paper
Abstract: Conversational agents in art exhibitions can enhance user engagement and understanding of artworks by providing contextual information, especially through voice interactions. However, creating a deeper personal connection with art - which often requires direct aesthetic and visual experiences - remains a challenge. This paper examines how integrating visual perception into conversational agents can enhance alignment with visitors' artistic interpretations, thereby fostering deeper engagement with interactive art exhibitions. We introduce a voice-based conversational agent enhanced with visual capabilities via a multimodal large language model (MLLM), allowing the agent to perceive, interpret and discuss artworks in real-time with visitors. The system utilizes a simplified Retrieval-Augmented Generation (RAG) architecture, which collects voice inputs, retrieves relevant information from a domain knowledge graph, and uses the LLM to generate conversational responses, which are then converted into voice outputs. A user study with 36 participants, divided into two groups, was conducted to compare the enhanced system with a baseline system that lacked visual input. Our results show that the visually enhanced system significantly improved visitor engagement and perception. Content analysis of the conversational transcripts further revealed a wider range of conversational topics, deeper visitor perceptions, and the agent's ability to provide more nuanced, visually-related discussions.
Institutions: University of Twente, Eindhoven University of Technology
Language: English
Subfield: Museology
Field: Arts and Humanities
Domain: Social Sciences
Open Access status: hybrid
OpenAlex ID: W4408614762