Comparative Performance Analysis of Large Language Models in Generative Business Intelligence: Insights from Llama3 and BambooLLM
Year: 2024
Type: article
Abstract: This study addresses the critical gap in Large Language Model (LLM) evaluation for business intelligence by conducting a rigorous comparative analysis of Llama3-70b-8192 and BambooLLM across five key data analysis tasks. Utilizing the AdventureWorks Cycle dataset, we developed a comprehensive evaluation framework measuring task efficiency, weighted accuracy, and misinterpretation rates. Results demonstrate that Llama3-70b-8192 outperforms BambooLLM with a 40% lower misinterpretation rate and 25% higher task efficiency across structured and interpretive business intelligence challenges. This study highlights the potential for optimizing fine-tuning strategies for task items that combine structured and interpretive elements, offering valuable insights for optimizing fine-tuning strategies and informing future research directions in LLM evaluation for business intelligence applications.
Authors: Bhavesh Jaisinghani -, Saurabh Aggarwal -
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4405979809