Indian Sign Language Interpretation Using Cnn and Mediapipe With Text-to-speech Integration
Year: 2025
Type: article
Abstract: : Indian Sign Language (ISL) is of great importance in communication for deaf and speechimpairedpeople from all around the nation of India. However, the lack of universally availableinterpretation tools has resulted in an obstacle to communication between ISL users and the generalpopulation. This research presents a real-time ISL interpretation system that consists of CNN andMediaPipe hand tracking to process gestures and convert them to natural language text. Finally, Textto-Speech (TTS) technology is integrated to allow for the output of the gestured information to hearingindividuals for supportive interaction. The proposed model takes in live video input, extracts handlandmarks based on MediaPipe output, and passes these landmarks into a trained CNN to make gestureclassification. Analytically, with high accuracy on a broad, machine-readable ISL dataset of commonlyused signs, the system achieves high accuracy. After classification, it translates the output into readabletext and converts it into speech to form a high-fidelity communication bridge between hearing andnon-hearing people. The way it integrates gesture detection, deep learning-based recognition, andspeech synthesis, is to enable real-time intuitive and efficient two-way communication. As alightweight, scalable system suitable for deployment on mobile and wearable devices, it largelyconforms to what is found in daily use as an educational, social, and professional tool. The systemshows great promise in improving social inclusion, accessibility, and independence of hearing andspeech-disabled individuals in India. Keywords: Indian Sign Language (ISL), Convolutional Neural Network (CNN), MediaPipe, Text-to-Speech (TTS), Gesture Recognition, Real-Time Communication, Assistive Technology.
Authors: Kaveri A. Chandan, Dr.Vijay B. More
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6911900454
FWCI:
Cites:
Cited by:
Related to: