Work

SilentSpeak: A Proposed Multilingual Deep Learning Frame work for Lip Reading in English, Hindi, and Marathi to Assist Individuals with Speech and Hearing Impairments

Year: 2026
Type: article
Abstract: Lip reading, or visual speech recognition, can recover spoken content when audio is missing, unusable or inaudible, which is valuable for people with hearing or speech impairments. Most published research targets Eng lish, and the authors' literature search found no publicly documented word-level lip-reading dataset or system for Hindi or Marathi. This paper proposes SilentSpeak, a conceptual framework for word-level lip reading in English, Hindi and Marathi that is designed to run on a standard webcam without an internet connection. The proposed pipeline locates the lips with MediaPipe Face Mesh, describes each frame with a small convolutional network, and models the mouth movement over time with a bidirectional long short-term memory network. An English base model trained on the GRID corpus is to be adapted to Hindi and Marathi through transfer learning, by freezing the convolutional layers and retraining the recurrent and output layers on small pilot vocabularies. The framework also proposes confidence scoring with a top-three fallback, an emergency-keyword alert and offline speech output. The paper reviews the rele vant literature, identifies the research gap, specifies the methodology and system architecture, and outlines a plan for evaluation. This is a proposed research direction, and no experimental results are claimed in this paper.
Language: English
Open Access status: green
OpenAlex ID: W7233755880
FWCI:
Cites:
Cited by:
Related to: 10