Hierarchical Temporal Modeling with Cross Stream Gating and Hybrid Neural Networks for Regional Sign Language Recognition
Year: 2026
Type: article
Abstract: SLR systems are the efficient and powerful devices to help deaf and hard-of-hearing individuals. The proposed architecture design is based on a mixed temporal encoder which includes different elements, such as separable Conv1D, bidirectional LSTM, and windowed self-attention using crossstream gating to connect 3D landmark kinematics by MediaPipe with optical flow by Farneback. The proposed architecture achieved 89.53% validation accuracy and 90.0% accuracy on an independent held-out test set, with an inference latency of 48 ms on the KSL dataset consisting of 1,287 videos spanning 11 gesture classes. This paper makes two significant contributions to applied research since it develops KSL research and creates an architecture that can be used in other languages in different countries with low financial and technological resources.
DOI: 10.53799/p8ttb781
Authors: Gurusiddappa Hugar, Ramesh M. Kagalkar
Institution: Nagarjuna College of Engineering and Technology
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7204850138