Get started
Work

Systematische Untersuchung: Finetuning von Sprachmodellen

Year: 2024
Type: dissertation
Abstract: (Draft uploaded by mistake, needs to be replaced with signed final version) The entire work is written in german, but here is a summary translated in english. ---- The subject of this work is the systematic investigation of fine-tuning language models.In this study, four models were examined: Llama2-7B-hf, microsoft-Phi-3-medium-4k-instruct, tiiuae-falcon-7b, and openai-community-gpt2. The models were quantized and fine-tuned with LoRA, utilizing various techniques. The aim of the investigation was to analyze the performance and the quality of responses on both general and domain-specific datasets, depending on the number of training epochs, and to gain insights into the fine-tuning process. Fine-tuning with LoRA shows that a higher LoRA rank and longer training can lead to the model forgetting (practically overwriting) more and faster of its pre-trained knowledge, while a smaller rank allows for better generalization. A disadvantage of this strategy is that the model learns more slowly.Since the focus was ultimately on the technical implementation and the details, the original goal was somewhat overlooked. Fine-tuning small language models does not necessarily require a large dataset or many epochs. However, it is important to review the choice of hyperparameters; the results, however, were only evaluated at the end of this work, leaving little room for adjustments. Moreover, the use of additional techniques and methods, such as FlashAttention, could increase the model's efficiency in terms of training and inference. Quantizing models and loading large models layer by layer allows them to be run on consumer-grade GPUs. However, this comes with an increased latency in inference and a degradation in model quality.
DOI: 10.5281/zenodo.15602777
Author: Reetz
Language: English
Open Access status: green
OpenAlex ID: W6893367753
Cites:
Cited by:
Related to: