Year: 2025
Type: dissertation
Abstract: In recent years, large language models (LLMs) have shown impressive results on various benchmarks, demonstrating their ability to produce contextually coherent responses to a wide range of queries. However, LLMs have notable limitations, including their reliance on static training data and their tendency to hallucinate. Retrieval-augmented generation (RAG) can address some of these limitations by combining a generative model with an external retrieval mechanism. This allows for grounded responses, reducing hallucinations, and supports updates of the knowledge base without requiring retraining. This thesis presents a proof-of-concept RAG-based AI assistant designed to retrieve technical vehicle manuals from an external document library. The AI assistant is implemented using the Azure RAG framework. To assess the model's trustworthiness, the proof-of-concept AI assistant, with tailored indexing and an improved generative model, was compared against a baseline model. Both models underwent manual and automatic evaluations, assessing their performance on shorter, simpler questions as well as longer, more complex ones. The improved model significantly outperformed the baseline in both evaluations, scoring 93 out of 100 compared to the baseline model's 79 out of 100 in the short-question evaluation. These results highlight the proof-of-concept AI assistant's potential for real-world applications.
Source: Aaltodoc (Aalto University)
Author: Talvitie, Sami
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7001889725