Research on a Speech-Modal Learning Behavior Evaluation Model Based on SenseVoice and LLMs
Year: 2026
Type: conference-paper
Abstract: With the evolution of smart education towards the stage of Multimodal Learning Analytics (MMLA), voice data holds significant value for the accurate evaluation of learners’ cognitive load and emotional engagement. This research proposes an automated evaluation framework based on dual-model collaboration, which indicatively integrates the emotional recognition capability of SenseVoice with the deep semantic understanding function of LLMs. By extracting rich emotion tags via SenseVoice and leveraging LLMs for high-order reasoning, the framework performs a multi-dimensional quantification of learners’ Semantic Content Representation (SCR), Engagement Features (EF) and Sentiment Orientation (SO). These metrics are then synthesized into a weighted Final Learning Score (FLS) to reconstruct high-fidelity learner profiles. Experimental validation across 50 group discussions in “black-box testing” scenarios reveals that the correlation coefficient between system-generated scores and expert ratings reached 0.897 (with the SCR dimension specifically hitting 0.911), while the Root Mean Square Error (RMSE) was 6.44. The results demonstrate that the multi-modal fusion model effectively mitigates the biases of single-metric evaluations through inter-dimensional “logical compensation.” It exhibits high scientific validity and robustness in simulating expert evaluation logic, thereby providing a reliable pathway for precise pedagogical intervention in digital environments.
Authors: Ting Kang, Yating Luo, Jie Zhang, Juan Li
Language: English
Subfield: Computer Science Applications
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: hybrid
OpenAlex ID: W7204934397