Bridging communication gaps: An AI-powered real-time system for sign language, speech and text translation
Year: 2025
Type: article
Abstract: Communication effectiveness poses an essential challenge to the millions who have hearing or speech impairments in their lives. HandSpeak provides a real-time AI interface through which users can interact easily because it transforms sign language into written messages and spoken words. The Sign-to-Speech module performs its functions through the integration of 3D cameras with Convolutional Neural Networks (CNNs) as well as Long Short-Term Memory (LSTM) networks to detect hand landmarks while tracking hand movements and interpreting sign gesture temporal sequences. Speech input flows through the Speech-to-Sign module to produce text output that gets processed into animated sign language expressions under AI avatar operations. Transformers boost linguistic precision and the integration between Django and Flask provides users with an optimized web-based interface. The application uses SQLite for optimized data storage together with Blender for producing sign animations. The technology serves to create an barrier-free environment for natural communication which enhances inclusivity while providing better accessibility to hearing and speech disabled people in various social and professional environments.
Institution: Aditya University
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
SDG: Reduced inequalities
Open Access status: hybrid
OpenAlex ID: W4408997773