Year: 2026
Type: conference-paper
Abstract: Despite the pivotal role of numerical reasoning as the cornerstone of mathematical capabilities in large language models (LLMs) across applications, few benchmarks evaluate LLMs by integrating numerical processing and mathematical reasoning, hindering the interpretability of failures in math tasks.We introduce PyraMath-Bench 1 , a comprehensive hierarchical benchmark with 32,505 questions derived from 7,404 math word problems, spanning 4 key cognitive aspects, 14 subcategories, and 2 modalities.Experiments reveal that LLMs' performance is severely compromised by inadequate numerical computation and weak handling of abstract numerical questions.To address this, we propose the Smart Optimization & Learning-based VErsatile module (SOLVE) and Interactive Relative Policy Optimization (IRPO), which enhance LLMs' numerical-mathematical synergy via efficient tool calls (fuzzy matching and low-quality call rejection).Comparative experiments show Qwen-2.5 achieves a 5.0 score improvement with SOLVE and IRPO training.* Corresponding author. 1 https://github.com/optifine233-ship-it/Key Units: 30 (meters^2) 22 (meters) To solve this MWP, I have to 1. Abstracting problems.2. extract key numbers and units, 3. solve a system of equations, … To solve this VRP, I have to 1. Abstracting problems.2. extract key numbers, 3. extract information from the picture, … L=6, W=5 or L=5, W=6 And L > W, so L=6 meters, W=5 meters The question can be abstracted as Unknown Variables: L,W Equations: 2*(L+W)=22, L*W=30 The question can be abstracted as 15 + 15 + 30 = ?60 / 2 = ?-(-11) ± (-11) 2 -4 * 30 2 = 𝟔, 𝟓 15 + 15 + 30 = 60 60 / 2 = 30 How many clocks are there in the picture? 3 Minutes of the digital clock: 15 Minutes of the analog clock 15, 30
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: G4748013168, G4976452659, G673242103
OpenAlex ID: W7166845561