Get started
Work

Implementation, Not Capabilities: How Chatbot Utilization Shapes Learners' Navigation Patterns in MOOCs

Year: 2025
Type: conference-paper
Abstract: Although massive open online courses (MOOCs) have broadened access to education worldwide, they often lack personalized support due to large enrollment and limited instructor capacity. Chatbots present an appealing solution by providing timely, on-demand assistance. However, few empirical studies have examined how different chatbot architectures, specifically intent-based versus large language model (LLM) retrieval-augmented generation (RAG) approaches, impact learner navigation patterns. To address this gap, we analyzed clickstream data from two journalist professional development MOOCs, each embedding a different chatbot type. The first course, an election reporting MOOC, incorporated an intent-based chatbot; the second, a digital content creation MOOC, employed an LLM-RAG chatbot. We analyzed the learners' clickstream data in two ways: measuring each event node's degree centrality and generating directed graphs to compare how learners moved through each course. Results showed that, while both chatbots ranked relatively high in node centrality, they did not dominate learners' navigation pathways. Certificate-related pages, announcements, and quizzes frequently attracted more attention in both courses. The LLM-RAG chatbot offered the potential for deeper personalization, yet its usage rates did not dramatically surpass those of the simpler intent-based chatbot. Our findings suggest that implementation strategies, such as integrating chatbots into core assessments and actively promoting their use, may play a greater role in shaping learner navigation than technological sophistication alone. We conclude that course designers aiming to optimize learner engagement with chatbots should focus on strategic deployment and alignment with course objectives rather than relying solely on advanced capabilities.
DOI: 10.5281/zenodo.15870214
Authors: Songhee Han, Hunhui Na, Chaewon Kim
Language: English
Open Access status: green
OpenAlex ID: W6949328532
FWCI:
Cites:
Cited by:
Related to: