Year: 2026
Type: preprint
Abstract: Agent Skills are structured packages of procedural knowledge that augment large language model (LLM) agents at inference time. Despite rapid adoption, there is no standard way to measure whether they actually help. We present SkillsBench, a benchmark whose current inventory contains 87 tasks across 8 domains paired with curated Skills and deterministic verifiers. Our latest aggregate evaluation runs the 87-task benchmark under matched no-Skills and curated-Skills conditions for 18 model-harness configurations. Curated Skills raise the average pass rate from 33.9% to 50.5% (+16.6 percentage points; 25.5% normalized gain), with configuration-level gains ranging from +4.1 to +25.7 pp. Focused Skills with at most three modules outperform larger or exhaustive bundles, and smaller models with Skills can match larger models without them. SkillsBench establishes paired evaluation as the foundation for rigorous measurement of Skill efficacy on agentic, expertise-heavy work.
Source: arXiv (Cornell University)
Authors: Li, Xiangyi, Liu, Yimin, Chen, Wenbo, You, Bingran, Di, Zonglin, He, Yifeng, Zheng, Shenghan, Choe, Kyoung Whan, Sun, Jiankai, Wang, Shuyi, Tao, Chujun, Li, Binxu, Zhao, Xuandong, Geng, Hejia, Wu, Xiaojun, Zhou, Junwei, Chen, Xiaokun, Xing, Hanwen, Li, Yubo, Zeng, Qunhong, Wang, Di, Wang, Yuanli, Chaim, Roey Ben, Jiang, Penghao, Shen, Haotian, Kong, Luyang, Liu, Xinyi, Wang, Runhui, Liu, Xuanqing, Li, Jiachen, Lan, Xin, Lin, Yueqian, Ye, Wengao, He, Junwei, Li, Songlin, Zhang, Yue, Gao, Yipeng, Li, Yijiang, Ma, Ze, Jing, Liqiang, Wang, Tianyu, Li, Kaixin, Xue, Yiqi, Lyu, Haoran, He, Yizhuo, Tian, Yuchen, Wu, Shutong, Wang, Bowei, Gao, Yixuan, Chen, Bo, Liu, Litong, Cheng, Sikai, Bao, Jiajun, Tong, Shuaicheng, Xu, Shuwen, Zhuo, Terry Yue, Ye, Tinghan, Qi, Qi, Li, Miao, Liao, Longtai, Tan, Zelin, Shi, Chang, Tang, Xilin, Tankasala, Srinath, Yuan, Boqin, Qian, Yaoyao, Tu, Jianhong, Wang, Chenguang, Sun, Yizhou, Wang, Wei, Taylor, Aaron, Yang, Ziyue, Guan, Changkun, Dong, Zhikang, Zhang, Xinyu, Dillmann, Steven, Lee, Han-chung, Song, Dawn
Institutions: Boston University, The Ohio State University, Amazon (United States), University of California, Berkeley, Carnegie Mellon University, University of California, Davis, Dartmouth College, Princeton University, Stanford University, University of California San Diego, The University of Texas at Austin, Michigan State University, Beijing Institute of Technology, The University of Texas at Dallas, University of California, Santa Cruz, Columbia University, University of Oxford, Duke University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7129079999