BiGRU-Based Bimodal Feature Fusion Model for English Listening and Speaking Recognition
Year: 2026
Type: conference-paper
Abstract: Due to the rapid development of AI in education, a new approach has been proposed to improve the ability of listening and speaking in higher education. In order to overcome the problems of traditional training, such as poor accuracy of speech recognition, delayed oral feedback, and poor multimodal coordination, this paper proposes a multimedia-assisted teaching model that integrates Bidirectional Gated Recurrent Units (BiGRU). It integrates two channels of speech image input, context awareness modeling, and feedback assessment mechanisms to create an integrated intelligent teaching lootion. Methodologically, Convolutional Neural Networks (CNNs) extract lip motion sequence features that are synchronized with speech features such as Mel-Frequency Cepstral Coefficients (MFCCs). BiGRU is a bidirectional temporal modeling approach that uses attention mechanisms to generate speech and image interactions. Training bias explicit annotation is achieved by using Softmax and heatmap feedback networks. Experimental results show that the proposed model is superior to traditional single-channel models in terms of semantic recognition accuracy, pronunciation clarity, rhythm stability and multimodal alignment loss. The results of this research demonstrate the feasibility and adaptability of the MBM in improving the efficiency of English listening and speaking training, which can be used as a reference for the design of Multimodal Teaching Interface (MTI).
Authors: Yixin Gao, Wenpu Wang
Institution: Chengdu Technological University
Language: English
Topic: Speech and Audio Processing
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: hybrid
OpenAlex ID: W7208729311