Sentimental Effects of Temperature Setting After QLoRA Fine-Tuning in LLAMA 2 7B and 13B models
Year: 2024
Type: dissertation
Abstract: This research investigates the impact of temperature settings and model size on the performance of AI language models, specifically focusing on LLAMA 2 7B and 13B models post-QLoRA fine-tuning. The study is driven by the need to understand how varying conditions affect these models’ abilities to replicate known answers. The methodology includes generating data and question-answer pairs using GPT-4, fine-tuning LLAMA 2 models, and then evaluating their performance using various statistical analyses. The research’s novel contribution lies in its comprehensive approach to evaluating AI language models under different conditions. It highlights the complexities in optimizing these models and underscores the importance of considering multiple factors in their deployment and operationalization. The findings provide valuable insights for future research, particularly in exploring diverse datasets and conditions to enhance model robustness and generalizability.
Author: Yildiz, Melih
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7015225953