Implementation of an AI Support Chatbot Based on Microsoft Azure OpenAI with Special Consideration of Quality
Year: 2025
Type: conference-paper
Abstract: This article describes the development and evaluation of an AI powered chatbot that was developed specifically to improve IT support at the IT-ServiceDesk (IT-SD) within the IT Center of RWTH Aachen University. The implementation is carried out using Microsoft Azure OpenAI and a Retrieval-Augmented Generation (RAG) approach. Given the varying complexity of support requests among different customer groups—namely, students, employees, and IT administrators—the AI chatbot will be a valuable supplement to established support channels such as email, telephone, chat, ticket portals, and personal interaction. The article describes the key requirements for the chatbot and explains how the quality of the response is ensured by a structured feedback system. It further addresses the challenges that arise from the fact that there are two different scenarios with different requirements. On one hand, standardized and publicly available RWTH support content is provided for RWTH members as well as interested parties of the IT services and, on the other, specific internal support content for the supporters at the IT-SD. This necessitates that specific measurable quality requirements and criteria must be identified implemented and adapted.
Source: EPiC series in computing
DOI: 10.29007/l6ss
Authors: Bernd Decker, Sarah Grzemski, Ingo Hengstebeck
Institution: RWTH Aachen University
Topic: AI in Service Interactions
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4415976503