Work

Advanced Hybrid Neural Networks for Accurate Recognition of the Extended Alphabet and Dynamic Signs in Mexican Sign Language (MSL)

Year: 2024

Type: article

Abstract: The Mexican deaf community primarily uses Mexican Sign Language (MSL) for communication, but significant barriers arise when interacting with hearing individuals unfamiliar with the language. Learning MSL requires a substantial commitment of at least 18 months, which is often impractical for many hearing people. To address this gap, we present an MSL-to-Spanish translation system that facilitates communication through a spelling-based approach, enabling deaf individuals to convey any idea while simplifying the AI’s task by limiting the number of signs to be recognized. Unlike previous systems that focus exclusively on static signs for individual letters, our solution incorporates dynamic signs, such as “k”, “rr”, and “ll”, to better capture the nuances of MSL and enhance expressiveness. The proposed Hybrid Neural Network-based algorithm integrates these dynamic elements effectively, achieving an F1 score of 90.91%, precision of 91.25%, recall of 91.05%, and accuracy of 91.09% in the extended alphabet classification. These results demonstrate the system’s potential to improve accessibility and inclusivity for the Mexican deaf community.

Source: Applied Sciences

DOI: 10.3390/app142210186

Authors: Arturo Lara-Cázares, Marco A. Moreno-Armendáriz, Hiram Calvo

Institution: Instituto Politécnico Nacional

Language: English

Topic: Hand Gesture Recognition Systems

Subfield: Human-Computer Interaction

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,728

Funder: Instituto Politécnico Nacional