Work

From Test-taking to Cognitive Scaffolding: A Pedagogical Diagnostic Benchmark for LLMs on English Standardized Tests

Year: 2026

Type: conference-paper

Abstract: As large language models (LLMs) are increasingly integrated into educational tools, current evaluations on standardized tests predominantly focus on binary outcome accuracy.Instead, an effective AI tutor must exhibit faithful reasoning, elucidate solution strategies, and diagnose specific human misconceptions.To bridge this gap, we introduce a pedagogical diagnostic framework that models English Standardized Test (EST) problem-solving as a traversal through a cognitive framework.Based on this framework, we present ESTBOOK, a multimodal benchmark encompassing 10,576 questions and 29 task types across five major exams.Unlike traditional datasets, EST-BOOK goes beyond data aggregation by enriching questions with formalized reasoning trajectories and distractor rationales that capture specific cognitive traps.Through extensive evaluations, we show that ESTBOOK identifies fine-grained reasoning bottlenecks and that lightweight mitigation based on these diagnoses provides evidence that the identified failure modes are actionable.< l a t e x i t s h a 1 _ b a s e 6 4 = " s / D e Z I p o j 8 3 w g J b q w Z M k t B 7 t / z 4 = " > A A A B 6 3 i c b V B N S 8 N A E J 3 4 W e t X 1 a O X x S J 4 K o l I 9 S I U v X i s Y D + g D W W z 3 b R L d z d h d y O G 0 L / g x Y M i X v 1 D 3 v w 3 b t o c t P X B w O O 9 G W b m B T F n 2 r j u t 7 O y u r a + s V n a K m / v 7 O 7 t V w 4 O 2 z p

Source: Conference on Empirical Methods in Natural Language Processing (EMNLP)

DOI: 10.18653/2026.emnlp-main.149