Year: 2025
Type: article
Abstract: We present an evaluation of ten state-of-the-art large language models (LLMs) taking the role of management consultants. The assessment is performed in a simulated interview setting where the LLMs are tasked with solving case studies – a popular testing procedure during job interviews at management consulting firms. We compose a dataset of eight annotated case studies with a total of 37 business questions and corresponding reference answers. We record the LLMs’ answers and evaluate their performance in terms of outcome and process validity. Our findings indicate that eight out of the ten evaluated LLMs would pass these case interviews, exceeding the minimum level of performance expected from human candidates. Further, the LLMs seem to take measures to ensure high-quality decisions. These findings highlight the potential of LLMs to serve as strategic advisors to managers in charge of making reliable decisions.
Authors: Malberg, Simon, Grigorev, Nikita, Rein, Victoria Lea, Borne Bass, Barbara, Gelvez Alvarez, Maria Alejandra, Groh, Georg
Institution: Technical University of Munich
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7054824880
FWCI:
Cites:
Cited by:
Related to: