Year: 2025
Type: dissertation
Abstract: This thesis explores the potential of Large Language Models (LLMs) in enhancing the Fleet Performance Review (FPR). The FPR is an analytical evaluation conducted by Wärtsilä Voyage. It involves a detailed examination of fleet data to generate actionable insights for decision-making. To gain an in-depth understanding of the FPR and identify areas where LLMs could enhance efficiency, a Contextual Inquiry was conducted. The Contextual Inquiry involved observing experts as they worked on the FPR. Findings indicated that typical tasks within the FPR involve the analysis of charts, maps and tables and the generation of actionable insights from these data types. It also highlighted pain points in how the FPR is currently operated, including how excessive expert time is spent on writing tasks and inconsistencies in subjective interpretations: issues that can potentially be mitigated with LLM assistance. A literature review was conducted to gain insights into the capabilities and challenges of state-of-the-art LLMs in comprehending charts, maps and tables, as well as insights relating to implementing LLMs into data analysis workflows. The review highlighted the promising capabilities of LLMs in chart and table comprehension, while noting the need for further research in map-related tasks. The study also identified user feedback of LLM usage in data analysis workflows as well as challenges such as output bias, hallucinations, and the need for effective prompt engineering. Based on the findings, a proof-of-concept (PoC) tool was developed leveraging GPT-4o to demonstrate the effectiveness of LLMs in enhancing a portion of the FPR workflow. A technical evaluation of the PoC revealed that the tool achieved a 97.7% recall rate for human-identified insights. A user test with 6 participants was conducted to gain qualitative insights into the integration of the tool into the FPR workflow, the findings of which revealed positive feedback on both the perceived effect on productivity, manual and cognitive effort of the participants, as well as the effect on the report quality.
Source: Aaltodoc (Aalto University)
Author: Malmsten, Herman
Institution: Aalto University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7064034633
