Hello, Freire! How Can You Help Me? A Smart Chatbot to Enhance Access to Academic Opportunities and Institutional Matters
Year: 2025
Type: conference-paper
Abstract: Context: Universities offer various resources and services that students often fail to utilize due to difficulties in accessing relevant information. Chatbots can improve access and interaction between students and institutions. Problem: Students struggle to find information about academic opportunities and institutional matters due to fragmented data, limiting engagement. Solution: This paper introduces Freire Assistant (Freire), a chatbot system powered by a Large Language Model (LLM) and built using Retrieval Augmented Generation (RAG) pipelines. Freire aims to streamline access to academic information through a fast, intuitive communication channel. IS theory: Thiswork was developed under the aegis of Soft Systems Theory, as it addresses the complexity of accessing academic information as a sociotechnical problem. Method: To evaluate Freire’s effectiveness in providing accurate and relevant information about academic opportunities and institutional matters, we conducted a case study at a Brazilian university. We quantitatively evaluated the contextual quality of Freire’s responses using the DeepEval framework. Additionally, we qualitatively evaluated Freire’s usability with 45 students through the Chatbot Usability Questionnaire (CUQ) which measures aspects related to Chatbot Personality, Onboarding, User Experience, and Error Handling. Summary of Results: The findings demonstrated Freire’s efficiency in generating contextually relevant and accurate responses. Furthermore, usability scores were generally high due to their simple interface and conversation-driven functionality. Contributions and Impact in the IS area: This study highlights how smart chatbots can enhance user experience and access to information in universities, providing a replicable model for other institutions facing similar challenges.
Authors: Arthur Willame Mesquita, Carlos Freire, Antônio Gomes, Bruna Amazonas, Anderson Uchôa
Institution: Universidade Federal do Ceará
Language: English
Topic: AI in Service Interactions
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: gold
Award: G8700604190
OpenAlex ID: W4410159990