Year: 2024
Type: conference-paper
Abstract: We present a novel study analyzing the effects of various prompt loss token weights (PLW) for supervised instruction fine-tuning (SIFT).While prompt-masking (PLW = 0) is common for SIFT, some fine-tuning APIs support fractional PLWs and suggest that using a small non-zero PLW can help stabilize learning when fine-tuning on short-completion data.However, there has never been a study confirming this claim, and OpenAI, a major cloud-based SIFT provider, recently removed this parameter from their fine-tuning API.We found that performance of models fine-tuned on shortcompletion data had a statistically-significant negative quadratic relationship with PLW.Using small values (0.01 -0.5) of PLW produced better results on multiple-choice and shortgeneration benchmarks (outperforming models fine-tuned on long-completion data) while large values (≈ 1.0) of PLW produced better results on long-generation benchmarks.We explained this effect and verified its importance through additional experiments.This research serves as a warning to API providers about the importance of providing a PLW parameter for SIFT.
Authors: Mathew Huerta-Enochian, Seung Yong Ko
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4404781823