Work

EREAL: Enhancing Mathematical Reasoning through Error-Aware Learning in Large Language Models

Year: 2026

Type: article

Abstract: Large Language Models (LLMs) have achieved substantial progress in solving complex mathematical problems, yet they often fail to exploit the potential value embedded in their own mistakes. Existing methods either apply coarse-grained corrections or rely on costly self-reflection, overlooking valid reasoning steps and incurring high inference overhead. To address these limitations, we propose Enhancing Reasoning through Error-Aware Learning (EREAL), a framework that enables fine-grained learning from edit-localized corrections. Our approach introduces minimal-edit correction, which revises only erroneous segments while preserving correct reasoning, and a dynamic error-aware loss that adaptively weights tokens based on error proportion. Experiments on seven mathematical reasoning benchmarks with instruction-tuned LLaMA-3.1-8B, DeepSeek-Math-7B, and Mistral-7B-v0.3 show consistent accuracy improvements. In particular, using only 4159 minimal-edit correction examples, the Mistral-7B-Instruct-v0.3 pipeline improves average accuracy by 23.9% relative to the original instruction-tuned baseline, while EREAL further delivers consistent gains over standard SFT trained on the same correction data. These results demonstrate the effectiveness of edit-localized supervision and the additional benefit of our error-aware learning objective. Our code is available at https://github.com/WHUIR/EREAL .

Source: Information Processing & Management

DOI: 10.1016/j.ipm.2026.105197

Authors: Yuqing Yu, Zihao Li, Lixin Zou, Chenliang Li, Qian Wang

Institution: Wuhan University

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

SDG: Quality education

Open Access status: hybrid

Estimated APC paid: $3,670