Evaluating Alignment Techniques for Enhancing LLM Performance in a Closed-Domain Application: A RAG Bench-Marking Study
Year: 2024
Type: conference-paper
Abstract: In the field of natural language processing (NLP), Large Language Models (LLMs) such as MISTRAL have been pre-trained on diverse tasks and datasets to achieve comprehensive language understanding. Nevertheless, fine-tuning small closed-domain datasets presents limitations, including overfitting, restricted generalization, data representation bias, and catastrophic forgetting issues. To tackle these challenges and align LLMs with ethical standards and specific requirements within closed-domain datasets, we apply a methodology that involves training four prominent LLMs - GEMMA, LLaMA, MISTRAL, and SOLAR - on a small closed-domain dataset using three distinct optimization techniques: Contrastive Preference Optimization (CPO), Direct Preference Optimization (DPO), and Kahneman-Tversky Optimization (KTO). We evaluate their proficiency by employing the Retrieval Augmented Generation (RAG) benchmark system to assess accuracy. Our research provides researchers with comprehensive insights into the effectiveness of these training methods in improving language model performance within closed-domain datasets while addressing the aforementioned challenges faced by LLMs in specific domains with limited datasets.
Authors: Ahmad Al-Zuraiqi, Des Greer
Institution: Queen's University Belfast
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4408146308