Year: 2025
Type: article
Abstract: : This work presents an advanced AI-based translation system designed to bridge communication barriers for the Deaf and Hard-of-Hearing (DHH) community by converting spoken and textual language into Indian Sign Language (ISL) and vice versa. The system leverages deep learning techniques, including computer vision and natural language processing (NLP), to interpret hand gestures and facial expressions accurately. Integrated with real-time processing capabilities, the model enables seamless interaction between ISL users and non-signing individuals. By utilizing a custom-trained Transformer-based NLP model and a Convolutional Neural Network (CNN) for visual recognition, the system ensures accurate and efficient translation. The prototype has been developed using VS Code, with datasets managed in local storage to optimize performance. This work aims to enhance accessibility, promote inclusivity, and facilitate effortless communication through a robust and scalable ISL translation model. The importance of an efficient ISL translation system extends beyond accessibility—it fosters independence, enhances social inclusion, and bridges the gap between the DHH community and the hearing population. Many Deaf individuals struggle with traditional text-based communication due to differences in sentence structures and grammar between ISL and spoken languages. By incorporating deep learning models for gesture recognition and NLP-based translation, our system provides a user-friendly solution for effective communication. Additionally, this system has the potential to be implemented in educational institutions, workplaces, and public services, ensuring better integration of the Deaf community into society. By addressing existing gaps and leveraging AI, our translator serves as a critical step toward an inclusive digital ecosystem.
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
SDG: Reduced inequalities
Open Access status: diamond
OpenAlex ID: W4408777505