Product
Help
Pricing
About
Jobs
Log in
Get started
Keyword
Add to collection
LLM benchmark
Description:
An LLM benchmark is a standardized evaluation task or dataset used to assess the capabilities of large language models.
Primary topic:
Topic Modeling (34%)
Related topics:
Topic Modeling (34%),
Artificial Intelligence in Healthcare and Education (9%)
Wikidata ID:
wiki/Q132127653
OpenAlex ID:
keywords/llm-benchmark
A review of large language models and autonomous agents in chemistry
2024
·
Mayk Caldas Ramos
, Christopher J. Collison
, et al.
·
Chemical Science
·
482
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack
2024
·
Yuri Kuratov
, Aydar Bulatov
, et al.
·
neural information processing systems
·
338
PDF
TheoremQA: A Theorem-driven Question Answering Dataset
2023
·
Wenhu Chen
, Ming Yin
, et al.
·
Conference on Empirical Methods in Natural Language Processing (EMNLP)
·
324
PDF
Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence
2025
·
Timothy R. McIntosh
, Teo Susnjak
, et al.
·
IEEE Transactions on Artificial Intelligence
·
301
PDF
ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation
2023
·
Du, Xueying
, Liu, Mingwei
, et al.
·
arXiv (Cornell University)
·
296
PDF
PentestGPT: An LLM-empowered Automatic Penetration Testing Tool
2023
·
Deng, Gelei
, Liu, Yi
, et al.
·
arXiv (Cornell University)
·
295
PDF
MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback
2023
·
Wang, Xingyao
, Wang, Zihan
, et al.
·
arXiv (Cornell University)
·
268
PDF
Holistic evaluation of large language models for medical tasks with MedHELM
2026
·
Suhana Bedi
, Hejie Cui
, et al.
·
Nature Medicine
·
266
Evaluating the Performance of Large Language Models on GAOKAO Benchmark
2023
·
Zhang, Xiaotian
, Li, Chunyang
, et al.
·
arXiv (Cornell University)
·
265
PDF
SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines
2025
·
P Team
, Du, Xinrun
, et al.
·
arXiv (Cornell University)
·
255
PDF
On the Tool Manipulation Capability of Open-source Large Language Models
2023
·
Xu, Qiantong
, Hong, Fenglu
, et al.
·
arXiv (Cornell University)
·
242
PDF
SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Research
2024
·
Liangtai Sun
, Yang Han
, et al.
·
Proceedings of the AAAI Conference on Artificial Intelligence
·
202
PDF
A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics
2023
·
He, Kai
, Mao, Rui
, et al.
·
arXiv (Cornell University)
·
189
PDF
Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinations
2023
·
Kasai, Jungo
, Kasai, Yuhei
, et al.
·
arXiv (Cornell University)
·
180
PDF
A Survey on Large Language Model Acceleration based on KV Cache Management
2024
·
Li, Haoyang
, Li, Yiming
, et al.
·
arXiv (Cornell University)
·
166
PDF
MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records
2024
·
Scott L. Fleming
, Alejandro Lozano
, et al.
·
Proceedings of the AAAI Conference on Artificial Intelligence
·
166
PDF
LLM4SR: A Survey on Large Language Models for Scientific Research
2025
·
Luo, Ziming
, Yang, Zonglin
, et al.
·
arXiv (Cornell University)
·
161
PDF
Natural Language Processing for Dialects of a Language: A Survey
2025
·
Aditya Joshi
, Raj Dabre
, et al.
·
ACM Computing Surveys
·
151
SuperCLUE: A Comprehensive Chinese Large Language Model Benchmark
2023
·
Xu, Liang
, Li, Anqi
, et al.
·
arXiv (Cornell University)
·
151
PDF
Harnessing Multiple Large Language Models: A Survey on LLM Ensemble
2026
·
Zhijun Chen
, Xiaodong Lu
, et al.
·
International Joint Conference on Artificial Intelligence
·
150
PDF
Open-LLM-Leaderboard: From Multi-choice to Open-style Questions for LLMs Evaluation, Benchmark, and Arena
2024
·
Myrzakhan, Aidar
, Bsharat, Sondos Mahmoud
, et al.
·
arXiv (Cornell University)
·
146
PDF
Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance
2023
·
Fu, Yao
, Ou, Litu
, et al.
·
arXiv (Cornell University)
·
144
PDF
MedCalc-Bench: Evaluating Large Language Models for Medical Calculations
2024
·
Nikhil Khandekar
, Qiao Jin
, et al.
·
neural information processing systems
·
130
Can Large Language Models Understand Real-World Complex Instructions?
2024
·
Qianyu He
, Jie Zeng
, et al.
·
Proceedings of the AAAI Conference on Artificial Intelligence
·
125
PDF
SafetyPrompts: A Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety
2025
·
Paul Röttger
, Fabio Pernisi
, et al.
·
Proceedings of the AAAI Conference on Artificial Intelligence
·
123
PDF
Show more works
View as search
Works count:
607
Citation count:
10,310
Year
Topic
Type