Product
Help
Pricing
About
Jobs
Log in
Get started
Keyword
Add to collection
agent evaluation benchmarks
Description:
Standardized tasks, datasets, or protocols used to assess the capabilities and performance of autonomous or AI agents.
Related topics:
AI-based Problem Solving and Planning (16%),
Artificial Intelligence Applications (9%),
Software Engineering Research (7%)
OpenAlex ID:
keywords/agent-evaluation-benchmarks
AgentBench: Evaluating LLMs as Agents
2024
·
Liu, Xiao
, Yu, Hao
, et al.
·
International Conference on Learning Representations
·
2,479
OpenHands: An Open Platform for AI Software Developers as Generalist Agents
2025
·
Wang, Xingyao
, Li, Boxuan
, et al.
·
International Conference on Learning Representations
·
1,086
PDF
Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models
2024
·
Zehui Chen
, Kuikun Liu
, et al.
·
Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
·
195
PDF
Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method
2024
·
Tian Xia
, Zhiwei He
, et al.
·
Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
·
83
PDF
OS Agents: A Survey on MLLM-Based Agents for General Computing Devices Use
2024
·
Xueyu Hu
, Tao Xiong
, et al.
·
Preprints.org
·
79
Mind the Sim2Real Gap in User Simulation for Agentic Tasks
2026
·
Zhou, Xuhui
, Sun, Weiwei
, et al.
·
arXiv (Cornell University)
·
77
PDF
Stop Comparing LLM Agents Without Disclosing the Harness
2026
·
Yunbei Zhang
, Janet Wang
, et al.
·
Preprints.org
·
57
PDF
MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering
2025
·
Rushi Qiang
, Yuchen Zhuang
, et al.
·
neural information processing systems
·
57
PDF
STEER: Assessing the Economic Rationality of Large Language Models
2024
·
Raman, Narun
, Lundy, Taylor
, et al.
·
RePEc: Research Papers in Economics
·
54
PDF
LLM-Based Data Science Agents: A Survey of Capabilities, Challenges, and Future Directions
2025
·
Rahman, Mizanur
, Bhuiyan, Amran
, et al.
·
arXiv (Cornell University)
·
39
PDF
WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web Tasks
2025
·
Miyai, Atsuyuki
, Zhao, Zaiying
, et al.
·
arXiv (Cornell University)
·
39
PDF
The Text-Based Adventure AI Competition
2019
·
Timothy Atkinson
, Hendrik Baier
, et al.
·
IEEE Transactions on Games
·
38
PDF
Foundations and Recent Trends in Multimodal Mobile Agents: A Survey
2024
·
Wu, Biao
, Li, Yanda
, et al.
·
arXiv (Cornell University)
·
36
PDF
Gentopia: A Collaborative Platform for Tool-Augmented LLMs
2023
·
Xu, Binfeng
, Liu, Xukun
, et al.
·
arXiv (Cornell University)
·
30
PDF
CToolEval: A Chinese Benchmark for LLM-Powered Agent Evaluation in Real-World API Interactions
2024
·
Zishan Guo
, Yufei Huang
, et al.
·
Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
·
29
PDF
KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation
2026
·
Chen, Tongbo
, Lu, Zhengxi
, et al.
·
arXiv (Cornell University)
·
28
PDF
The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines
2026
·
Vaishali Vinay
·
IEEE International Conference on AI in Cybersecurity (ICAIC)
·
27
PDF
Large Language Model Agents for Investment Management: Foundations, Benchmarks, and Research Frontiers
2025
·
Preetha Saha
, Jingrao Lyu
, et al.
·
ACM International Conference on AI in Finance (ICAIF)
·
25
PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents
2026
·
Xue, Shuhan
, Ding, Zixin
, et al.
·
arXiv (Cornell University)
·
20
PDF
VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactions
2026
·
Chen, Yuxin
, Zhang, Yi
, et al.
·
arXiv (Cornell University)
·
17
PDF
A Framework for Studying AI Agent Behavior: Evidence from Consumer Choice Experiments
2026
·
Cherep, Manuel
, Ma, Chengtian
, et al.
·
International Conference on Learning Representations
·
14
PDF
Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents
2026
·
Muyu He
, Anand Kumar
, et al.
·
Proceedings of the conference - Association for Computational Linguistics. Meeting
·
14
PDF
Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures
2026
·
Raj, Harsh
, Gupta, Vipul
, et al.
·
arXiv (Cornell University)
·
13
PDF
PrefIx: Understand and Adapt to User Preference in Human-Agent Interaction
2026
·
Li, Jialin
, Chen, Zhenhao
, et al.
·
arXiv (Cornell University)
·
13
PDF
Evaluating Long-Context Reasoning in LLM-Based WebAgents
2025
·
Chung, Andy
, Zhang, Yichi
, et al.
·
arXiv (Cornell University)
·
12
PDF
Show more works
View as search
Works count:
110
Citation count:
4,867
Year
Topic
Type