Translating Hands Gestures into Text and Speech
Year: 2025
Type: article
Abstract: Significant progress has been made in hand sign gesture recognition systems as a result of the growing need for inclusive communication solutions. This study offers a comprehensive framework for converting hand sign movements into speech and text by utilizing Dense Neural Networks (DNNs) in conjunction with cutting-edge deep learning and computer vision technologies. The suggested method ensures accurate gesture recognition by combining OpenCV with MediaPipe for real-time hand tracking and landmark detection. A strong DNN that can correctly identify gestures is designed and trained using TensorFlow and Keras, and the identified motions are then translated into speech that sounds natural using GTTS (Google Text-to-Speech). The Tkinter (Tk) toolkit was used to create an intuitive graphical user interface that facilitates accessibility and smooth interaction. The procedure uses MediaPipe and OpenCV to preprocess input video streams in order to extract hand landmarks. After being normalized, these landmarks are entered into the DNN model for categorization. The resultant text output is simultaneously translated into speech using GTTS and shown on the interface. Results from experiments show that the system can recognize both static and dynamic gestures with great accuracy in a variety of background and lighting situations. By offering a portable, real-time, and affordable alternative for sign language interpretation, this research helps close the communication gap for those with speech and hearing impairments.
Authors: Patil, Kunika, Ladake, Sakshi, Nirgude, Shraddha, Naphade, Vaishnavi, Thummalakunta, Prof. (Dr) Praveen
Institution: Zeal College of Engineering and Research, Pune
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
SDG: Reduced inequalities
Open Access status: green
OpenAlex ID: W6930628576
FWCI:
Cites:
Cited by:
Related to: