Large Language Models as an aid for software development using graphical programming language
Year: 2023
Type: dissertation
Abstract: This thesis explores the use of Large Language Models (LLMs) as a tool for software development in a graphical programming language, specifically LabVIEW.The thesis aims to investigate the properties of different LLMs and determine their effectiveness on LabVIEW programming.The research questions addressed in this thesis are: 1) How can different LLMs be used as a tool for software development with a graphical programming language like LabVIEW, 2) How does tuning LLM with prompts affect the help it provides, 3) What kind of issues software developers may confront when they exploit LLMs in graphical programming, and 4) How the quality of help provided by LLMs can be evaluated?The findings of this research aim to deepen the understanding of LLMs and identify potential LLMs to be used by automation application developers.The thesis begins with a theory section covering the basic idea of LLMs and a literature review of published models, which is followed by experimental research, where models are evaluated using a collection of questions related to LabVIEW programming and debugging.Most of the recently published models are based on transformer architecture, while having differences in the hyperparameters and amount of training data used.Some of the reviewed models were integrated with web applications such as OpenAI's ChatGPT and Google's Bard, while other models have been publicly published to the Hugging Face platform, which can be downloaded on users' machines.Models chosen for experimental research were GPT-3.5 and GPT-4 in ChatGPT, GPT-3.5-turbo in OpenAI's API, and PaLM 2 in Bard.The topics of the experimental study were divided into three cases: software design, model tuning using images, and error correction using image prompts.The questions were designed to simulate a conversation between the user and the LLM.The evaluation of the questions and the model responses were documented in an evaluation form.The results show that the customized GPT-4 model LabVIEW-GPT in ChatGPT could be used as a tool for LabVIEW programming, as it worked best, especially for image analysis of LabVIEW code.PaLM 2 is a great free replacement for the GPT-4 model, as its performance is close to GPT-4 in text prompts.For image analysis of LabVIEW code, PaLM 2 performs poorly.The GPT-3.5 model performed poorly on most questions, so it cannot be used as a LabVIEW programming tool.While the models can be used as LabVIEW programming tools, they still have many limitations that lower their usability.Models tend to hallucinate, misalign between the user and the model, and web applications can sometimes have poor usability.Thus, more traditional information retrieval methods, such as search engines, may be more effective than LLMs.
Author: Tiljander, Ville
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7133042046