SIGN-AI: A Unified Framework for High-Fidelity Sign Language Synthesis via Text, Voice, and Video Inputs
Year: 2025
Type: article
Abstract: This paper introduces SIGN-AI, a transformative framework for real-time sign language synthesis addressingcritical accessibility gaps for 70 million deaf individuals worldwide. Confronting significant barriers—including 83%lack of accessible STEM education materials, only 4.7% online content with sign interpretation, and 3× higherunemployment rates—we propose two novel algorithmic approaches: Algorithm 1 (Text/Image-Driven Synthesis)employs a modular pipeline converting multimodal inputs to text, generating sign images via Vision Transformerstrained on SGN-IMG datasets, animating frames through optical flow prediction (0.5-2s clips), and sequencing withLSTM models. This approach delivers computational efficiency (CPU-feasible) and explicit sign control. Algorithm 2(End-to-End Video Synthesis) utilizes diffusion models trained on SGN-VID datasets to directly translate text intomotion vectors, producing 1-3s sign segments with attention-based fusion, achieving superior motion fidelity throughlearned co-articulation.Benchmarks demonstrate 70% resource reduction (Algorithm 1) and 92% motion naturalness(Algorithm 2) versus state-of-the-art. Projected impacts include: 45% faster STEM concept acquisition, 30% increasein deaf graduates by 2035, and $17B annual economic productivity gain. The framework enables real-time translationof streaming media, educational content, and professional communications, with future research targeting low-latencyvideo-to-video conversion and holographic rendering.
Author: Guechi Ayoub
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
SDG: Reduced inequalities
Open Access status: green
OpenAlex ID: W6931193883
FWCI:
Cites:
Cited by:
Related to: