Work

MSA at BEA 2025 Shared Task: Disagreement-Aware Instruction Tuning for Multi-Dimensional Evaluation of LLMs as Math Tutors

Year: 2025

Type: conference-paper

Abstract: We present MSA-MATHEVAL, our submission to the BEA 2025 Shared Task on evaluating AI tutor responses across four instructional dimensions: Mistake Identification, Mistake Location, Providing Guidance, and Actionability.Our approach uses a unified training pipeline to fine-tune a single instructiontuned language model across all tracks, without any task-specific architecture modifications.To improve prediction reliability, we introduce a disagreement-aware ensemble inference strategy that enhances coverage of minority labels.Our system achieves strong performance across all tracks, ranking 1 st in Providing Guidance, 3 rd in Actionability, and 4 th in both Mistake Identification and Mistake Location.These results demonstrate the effectiveness of scalable instruction tuning and disagreementdriven modeling for robust, multi-dimensional evaluation of LLMs as educational tutors.

Source: Workshop on Innovative Use of NLP for Building Educational Applications

DOI: 10.18653/v1/2025.bea-1.95

Authors: Baraa Hikal, Mohamed Basem, Islam Oshallah, Ali Hamdi

Institution: October University of Modern Sciences and Arts

Language: English

Topic: Intelligent Tutoring Systems and Adaptive Learning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

SDG: Quality education

Open Access status: hybrid

OpenAlex ID: W4412889161