Year: 2024
Type: conference-paper
Abstract: Effective communication with the deaf and hard-of-hearing community remains a significant challenge due to the complexities of translating spoken and written language into sign language. This paper introduces a novel real-time text-to-sign language conversion system utilizing advanced Natural Language Processing (NLP) technologies, including transformers and BertTokenizer. Development details include a WebGL-based user interface facilitating seamless interaction with a 3D humanoid character performing accurate sign language animations. The methodology employs robust tokenization, efficiently mapping text to corresponding sign gestures, enhanced by synonym matching and part-of-speech filtering to accommodate linguistic variations. Evaluation using the MS-ASL and WordNet datasets achieved a translation accuracy of 96.8%. This study advances technological frameworks assisting communication with the deaf community and sets the stage for future enhancements, such as integrating facial expressions and expanding dataset dimensions to cover more linguistic features.
Authors: Sara Hridita, Musfiqur Rahman, Md Abu Talha, Abdul Al Mahmud Riaz, Safwan Ahmad Chowdhury, Khandaker Tabin Hasan
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
SDG: Reduced inequalities
Open Access status: bronze
OpenAlex ID: W4411097622