Estudio comparativo de técnicas PEFT sobre modelos LLM para la generación de SQL a partir de lenguaje natural en entornos con recursos limitados
Year: 2025
Type: dissertation
Abstract: This study presents a comparative evaluation of parameter-efficient fine-tuning techniques (PEFT) applied to large language models (LLMs), with an emphasis on their performance for the specific task of generating SQL queries from natural language (NL-to-SQL) in computationally constrained environments. Four prominent PEFT techniques are analyzed: LoRA, QLoRA, Prefix Tuning, and (IA)³, using GPT-2 and LLAMA-7B as base models. The evaluation considers both aspects related to computational efficiency (training time, GPU memory consumption, and resource utilization) and performance metrics in the NL-to-SQL translation task (loss function in training and evaluation, convergence, generalization, and inference times). This analysis concludes that techniques such as QLoRA and LoRA constitute preferable solutions for computationally constrained environments, providing robust and efficient results, standing out for their low GPU memory consumption and fast convergence in the task of generating SQL queries from natural language. On the other hand, Prefix Tuning and (AI)³ present significantly longer training times and require additional tuning to achieve comparable performance, making their adoption less viable in the described scenario without a deeper hyperparameter tuning.
Language: Spanish
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7158435398