Year: 2023
Type: conference-paper
Abstract: The remarkable capabilities of Natural Language Models to grasp language subtleties has paved the way for their widespread adoption in diverse fields.However, adapting them for specific tasks requires the time-consuming process of fine-tuning, which consumes significant computational power and energy.Therefore, optimizing the fine-tuning time is advantageous.In this study, we propose an alternate approach that limits parameter manipulation to select layers.Our exploration led to identifying layers that offer the best trade-off between time optimization and performance preservation.We further validated this approach on multiple downstream tasks, and the results demonstrated its potential to reduce fine-tuning time by up to 50% while maintaining performance within a negligible deviation of less than 5%.This research showcases a promising technique for significantly improving fine-tuning efficiency without compromising task-or domain-specific learning capabilities.
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4389518277