Retrieval-Augmented Generation for Industrial Applications: Design and Evaluation of an AI Assistant
Year: 2026
Type: dissertation
Abstract: Industrial machinery manufacturers distribute technical knowledge through dense manuals that operators and engineers must navigate manually to answer operational questions.This process is time-consuming, prone to error, and inaccessible to users without prior familiarity with the documentation.Large language models offer a promising alternative, but their reliance on static parametric knowledge makes them unsuitable for domain-specific technical queries where safety, accuracy and grounding in authoritative sources are critical.This dissertation presents AMOB-RAG, a modular Retrieval-Augmented Generation system designed to answer natural language questions in Portuguese about technical documentation for rotary draw bending machines.The system is structured around two pipelines: an offline ingestion pipeline that preprocesses, chunks, and indexes markdown documentation into a persistent vector store, and an online query pipeline that retrieves, reranks, and assembles context before generating grounded answers.Key design decisions include semantic chunking with configurable breakpoint detection, CrossEncoder reranking with a table penalty heuristic, domain-specific query expansion from a structured axis synonym file, and a similarity threshold gate that prevents low-confidence retrievals from reaching the generation stage.The system is evaluated against a human-verified golden dataset of 20 question-answer pairs in Portuguese, covering five question categories.Evaluation combines automated metrics with an LLM-as-a-judge framework that scores each answer across three qualitative dimensions: faithfulness, answer relevance, and context relevance.A controlled experiment strategy isolates the contribution of each pipeline component.A baseline experiment demonstrates that a standalone large language model fails to answer domain-specific questions reliably, establishing the necessity of the RAG architecture.Subsequent controlled experiments validate the individual contributions of each ingestion and query pipeline component.The results, obtained from this initial case study, confirm that the modular architecture achieves reliable retrieval and grounded answer generation over specialized industrial documentation, with the evaluation framework providing sufficient diagnostic resolution to identify failure modes in specific pipeline components.As the system has not yet been deployed within AMOB's production environment, these findings establish a validated technical foundation ahead of a future field deployment and evaluation with AMOB's operators.
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7202060095