Year: 2024
Type: article
Abstract: Due to the fast advancement in large language models (LLMs), the natural language processing (NLP) technology domain has witnessed a massive change. It has brought ground breaking developments in how machines understand and generate human language. However, even with these advancements, it remains hard to objectively compare and evaluate LLMs' performance because various benchmarks and metrics are used in the assessment. Such a method requires this study's writers to conduct a comparative analysis of various benchmark LLMs to explain the viability of different evaluation metrics. Analyzing the recognized standards, including GLUE, SuperGLUE, and SQuAD, reveals weaknesses and potential in the present evaluating systems. It embraces quantitative assessments of model performances and benchmarking and critical evaluations of benchmark designs and their scope. The paper discusses methodological issues in benchmarking research and explains that current measures are insufficient and lack comprehensive evaluation patterns. The conclusions of this research underline the significance of creating fully integrated and efficient reference sets that match the rate of innovation in LLM ability and inform other studies and the overall subsequent advancement of more sophisticated and flexible language models.
Authors: Valentina Porcu, Aneta Havlínová
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4404419427