Work

Sign Language Recognition Using Enhanced Feature Extraction and Classification Using a Hybrid Architecture

Year: 2026
Type: preprint
Abstract: Sign language conveys meaning through both manual gestures (hand shapes and movements) and non-manual cues such as facial expressions and body posture, yet previous studies have often modeled these channels separately, which can lose meaning. This study proposes a hybrid deep learning architecture that recognizes manual and non-manual gestures jointly. Each video is divided into 16-frame segments; YOLOv8 localizes the face, VGG16 and a recurrent neural network extract spatial and temporal facial features, and an improved 3D convolutional network (C3D) captures full-body spatiotemporal motion. The fused features are classified by an improved bidirectional long short-term memory network (iBiLSTM). The framework was evaluated on three benchmark datasets, PkSLMNM, RWTH-PHOENIX-Weather 2014T, and SILFA, using 5-fold cross-validation. On PkSLMNM, the model achieved a mean cross-validation accuracy of 95.79% (standard deviation 0.37%), outperforming LSTM (86.12%) and BiLSTM (89.15%) baselines, and it reached 93.84% and 94.21% accuracy on RWTH-PHOENIX-Weather 2014T and SILFA, respectively. An ablation study showed that fusing facial and body features outperformed either modality alone. These results indicate that combining manual and non-manual cues in a unified architecture improves both the accuracy and the cross-dataset robustness of dynamic sign language recognition.
Language: English
Open Access status: green
OpenAlex ID: W7236793144
Cites:
Cited by:
Related to: 10