Work

AI-driven Augmentative and Alternative Communication System for Communication Enhancement in Cerebral Palsy Using American Sign Language Recognition

Year: 2025

Type: article

Abstract: Augmentative and alternative communication (AAC) methods are designed to facilitate effective communication for individuals with speech, language, or writing impairments.We introduce a novel AAC system designed to enhance communication for individuals with speech, language, or writing impairments, such as those with cerebral palsy (CP).Addressing the limitations of existing AI-driven AAC solutions, advanced AI and machine learning techniques are integrated in the system to minimize user effort in conveying their thoughts.Utilizing a readily available webcam as a key input modality, the system employs MediaPipe to capture hand gestures corresponding to American Sign Language signs.The resulting visual data is then processed and classified by a random forest model.By interpreting these sensor-captured gestures, the system enables users to input partial vocabulary, which subsequently prompts generative AI models to predict and complete intended text.Empirical evaluations, conducted through two distinct experiments, validate the system's viability and demonstrate its potential to significantly improve communication accessibility for individuals with CP through an accessible and intuitive gesture-based interface.

Source: Sensors and Materials

DOI: 10.18494/sam5710

Authors: Yuh-Shihng Chang, Chen-Wei Lin, Chien-Chih Chen

Institution: National Chin-Yi University of Technology

Language: English

Topic: Assistive Technology in Communication and Mobility

Subfield: Occupational Therapy

Field: Health Professions

Domain: Health Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $748