Análisis de dades sobre el progrés en models de llenguatge
Year: 2024
Type: dissertation
Abstract: [EN] Large language models or LLMs are transforming the artificial intelligence sphere, causing an unprecedented investment, but it is difficult to know objectively how they behave in different fields and how to compare them with their new versions. This work addresses the solutions proposed by the community in the form of bench- marks to evaluate these models, showing real examples and results, following the progress since their beginnings. We further explore how these test sets are constructed, with the different disciplines to be evaluated and the metrics used. Then we propose as a case study the benchmark BIG-Bench, analyzing the results obtained for two main LLMs fam- ilies (GPT and Big-G) in the different tasks used for their evaluation, focusing on the performance trends they show as the trainable parameters increase. They are also com- pared with results obtained by humans in the same tests. We set as a main objective to find disciplines in which these models still have difficulties and their score does not scale with the number of parameters.
Source: Hispana
Author: Feliu Descalzo, Miguel
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7148198523