Work

Tree-of-thoughts kehotesuunnittelu viitekehys tilinpäätösanalyysille

Year: 2025
Type: dissertation
Abstract: Large Language Models are increasingly capable of complex tasks and advanced reasoning. Use cases have been suggested for many high-stakes fields, such as law, medicine and finance. However, deploying these models for real world analytical tasks is still difficult and risky, as there are many considerations for model performance and result evaluation. Tasks also vary in context attributes, as for example legal documents often consist mainly of text data. Therefore, domain specificity is an important consideration, for example with business analytics tasks, where the context often consists mainly of structured numerical data. This research aims to develop a prompting framework specifically for business analytics domain, and it uses a set of financial statements data from 110 Finnish SMEs to test the analytical consistency and performance of the OpenAI o3 model with a series of prompts compiled according to that framework. The results indicate that at best the analytical performance of the current models could be useful for real-world uses with well-developed prompt for the specific task. However, the results clearly show that human evaluation of the results cannot be bypassed when using the models. This research emerged from the desire to develop low-threshold ways to adopt LLMs for recurrent and clearly structured analytics tasks, and to develop similarly low-threshold ways to evaluate and validate the model performance. The results of the thesis form a decent basis for such purposes.
Author: Bragge, Joonas
Language: English
Open Access status: green
OpenAlex ID: W6987831851
Cites: 68
Cited by:
Related to: 10