Work

Revisiting Instruction Fine-tuned Model Evaluation to Guide Industrial Applications

Year: 2023

Type: conference-paper

Abstract: Instruction Fine-Tuning (IFT) is a powerful paradigm that strengthens the zero-shot capabilities of Large Language Models (LLMs), but in doing so induces new evaluation metric requirements.We show LLM-based metrics to be well adapted to these requirements, and leverage them to conduct an investigation of taskspecialization strategies, quantifying the tradeoffs that emerge in practical industrial settings.Our findings offer practitioners actionable insights for real-world IFT model deployment.

Source: Conference on Empirical Methods in Natural Language Processing (EMNLP)

DOI: 10.18653/v1/2023.emnlp-main.559

Authors: Manuel Faysse, Gautier Viaud, Céline Hudelot, Pierre Colombo

Institutions: Université Paris-Saclay, CentraleSupélec, Mathématiques et Informatique pour la Complexité et les Systèmes

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: Association Nationale de la Recherche et de la Technologie, Grand Équipement National De Calcul Intensif

OpenAlex ID: W4389520751