Year: 2026
Type: preprint
Abstract: AI agents are playing an increasingly important role in physics research. However, existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness. We introduce FrontierPhysics, an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains across theoretical, computational, and experimental physics. Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts. Combining deterministic verifiers with expert-authored rubrics, FrontierPhysics evaluates whether agents' research is correct and publishable. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts. Manual review of agent trajectories shows that frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors.
Source: arXiv (Cornell University)
Authors: You, Bingran, Ke, Wenjun, Rosenauer, Ria, Liu, Bo, Chen, Wenbo, Chen, Min, Xu, Shuqi, Ma, Yue, Feng, Jiechao, Chen, Bo, Sun, Xiangkai, Liu, Jiarui, Guo, Jinen, Zhou, Zeyu, Huang, Xu, Li, Zhuohan, Li, Haoyu, Lin, Heguang, Ding, Lijie, Zeng, Yadong, Zhang, Kangqi, Liu, Xinge, Liu, Hanwen, Li, Zhenghong, von Recum, Alexander, Zhu, Yuchen, Schwarzfischer, Felix, Zhao, Yizhou, Wang, Jicheng, Li, Binxu, Yu, Qian, Li, Chan, Dai, Cheng, Wang, Di, Zhang, Haoyu, Sun, Jiankai, Feng, Kaituo, Qin, Mian, Yin, Ruihong, Luo, Yiqun, Wu, Weiwei, Pan, Shihong, Hsieh, Tsung Yeh, Huang, Xiao, Zhang, Yan, Li, Sihan, Ren, Zirui, Tu, Zengji, Choe, Kyoung Whan, Sun, Ke, Galia, Vincenzo, Mishra, Vaisakhi, He, Jiashu, Wang, Hongyang, Yang, Zhiheng, Gaur, Shubham, Gu, Yifan, Wang, Shuyi, Zuo, Zihao, Dillmann, Steven, Sun, Yiyou, Han, Xinyang, Lee, Hanchung, Tao, Molei, Zhao, Yue, Liu, Shilong, Wu, Yuchen, Zhao, Xuandong, Li, Xiangyi, Brand, Florian, Ceder, Gerbrand, Haeffner, Hartmut
Institutions: University of California, Berkeley, California Institute of Technology, Harvard University, Amazon (United States), University of Washington, Stanford University, University of Pittsburgh, University of California San Diego, Northeastern University, Georgia Institute of Technology, Lawrence Berkeley National Laboratory, University of Michigan, Scripps Research Institute, Oak Ridge National Laboratory, The University of Texas at Austin, University of Toronto, University of Southern California, Ludwig-Maximilians-Universität München, Princeton University, University of Minnesota, New York University, Nanyang Technological University, University of Palermo, University of California, Davis, University of Amsterdam
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225298314