Avoimen lähdekoodin suurten kielimallien hienosäätö avointen komentojen tulkitsemiseen robottinavigoinnissa
Year: 2025
Type: dissertation
Abstract: This thesis investigates using fine-tuned open-source Large Language Models (LLMs) for interpreting open-vocabulary commands for robotic navigation tasks. In this study, this means retrieving objects from scene graphs based on freeform language instructions. This thesis seeks to address the following research questions: To what extent are open-source LLMs suitable for open-vocabulary tasks? How does model size influence performance? How effective are fine-tuned models when compared to their original, unmodified counterparts? For this study, four different sized LLMs distilled from DeepSeek-R1 model were chosen. These chosen models were then fine-tuned on a training set created from data selected from VLA-3D dataset collection. During the fine-tuning process, a hyperparameter search was conducted. The best fine-tuned models for each size were then tested on another set created from VLA-3D dataset collection and statements from ViGiL3D-dataset. It was observed that fine-tuned models improve significantly on statements that follow the style of statements included in VLA-3D dataset collection. When tested on more free-form statements from ViGiL3D-dataset, the improvements were less drastic but still notable. The results indicate that open-source LLMs can be used in the task presented in this thesis. The fine-tuned models achieved better results compared to the original models. It was also observed that in some cases smaller model could outperform bigger model.
Source: Aaltodoc (Aalto University)
Author: Palmulaakso, Joonas
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7076969766