Work

Interpretable and lightweight multimodal Transformer assessment of English education ecosystem health

Year: 2026

Type: article

Abstract: How can the health of an English classroom ecosystem be assessed from routine recordings in a way that is accurate, traceable to observable behaviour and inexpensive enough to run inside a school? The question is harder than it looks, because what is taught, how it is delivered and how learners respond are dynamically coupled: the difficulty of the material shapes delivery, and delivery in turn shapes learner behaviour. Single-modality models and shallow feature concatenation capture only fragments of that interplay. To answer the question, we build a multimodal Transformer that encodes textual, acoustic and visual streams through modality-specific front ends, learns their mutual dependence through bidirectional cross-modal attention, and merges them through a learned gate to predict a five-dimensional Ecosystem Health Index whose scoring rubric, rater protocol and entropy-based weighting are fully specified. A gradient-weighted attribution scheme turns attention into modality-decomposed, temporally localised explanations, which show textual features dominating healthy sessions and video-based disengagement cues taking over in degraded ones. A joint distillation and structured pruning pipeline compresses the network from 23.8 M to 2.9 M parameters, an 8.2-fold reduction that retains 88.2% of the teacher model’s RMSE-based performance. On 6,142 sessions recorded in 20 institutions the full model reaches an RMSE of 0.0431 ± 0.0009 and a coefficient of determination of 0.923 ± 0.004 across five random seeds, ahead of unimodal, early-fusion and prior cross-modal baselines under paired significance testing. Error rises by 13.5% to 34.1% once instructors, institutions, provinces, course types or educational levels are held out, and we treat that gap as the honest boundary of the present evidence rather than as noise. Device-level benchmarks on Jetson Nano and Raspberry Pi hardware, together with an explicit responsible-use protocol that rules out punitive teacher evaluation, complete the account.

Source: Scientific Reports

DOI: 10.1038/s41598-026-77249-3

Authors: Meiqin Lv, You Wang, Dong Han

Institutions: Wuhan Vocational College of Software and Engineering, Wuhan Open University, Jilin University, Northeast Normal University

Language: English

Topic: Educational Technology and Pedagogy

Subfield: Education

Field: Social Sciences

Domain: Social Sciences

SDG: Quality education

Open Access status: gold

Estimated APC paid: $2,690