Year: 2026
Type: article
Abstract: To address the common issues in existing lip-reading recognition systems, such as insufficient feature extraction and visual ambiguities, a Chinese sentence-level lip-reading recognition system based on residual networks and attention mechanisms (Spatio-Temporal Cascade Network, STCNet) is proposed. This system can first translate continuous videos containing speakers into pinyin sequences sentence by sentence, and then convert them into continuous Chinese sentences. Firstly, the residual network in the front-end feature extraction module can achieve bidirectional transmission and fusion of multi-level features, enhancing the extraction capability of long-term temporal features. In the back-end pinyin classification module, a Temporal Convolutional Network (TCN) is used to extract long-term dependency information, and an attention mechanism is introduced to weight discriminative features at different time intervals to reduce visual ambiguities
Authors: Chuan Qin, ZiLong Liu
Institution: University of Shanghai for Science and Technology
Language: English
Topic: Speech and Audio Processing
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7236861317
