Work

Accessible American Sign Language Learning in Virtual Reality via Inverse Kinematics

Year: 2025

Type: article

Abstract: Along with the rapid advancement of Virtual Reality (VR) and the metaverse, interest in this technology has surged among game developers and in fields such as education and healthcare. VR has enabled the rise in immersive, gamified activities, whether for rehabilitation, therapy, or learning. Additionally, VR and Motion Capture (MoCap) have allowed developers to create further accessibility features for end-users with special needs. However, the excitement of using new technology often does not align with the end user’s use cases. The over-reliance on cutting-edge hardware can negatively impact most end users who lack access to such expensive tools. To this end, we conducted an inclusivity-focused study that enables learners to practice ASL in an immersive and engaging way using only head- and controller-based tracking. Our approach replaces full-body MoCap with Inverse Kinematics (IK) and simple controller mappings for upper-body pose and hand-gesture recognition, providing a low-cost, reproducible alternative to costly setups.

Source: Virtual Worlds

DOI: 10.3390/virtualworlds4040057

Authors: Jeremy Immanuel, Santiago Berrezueta-Guzman

Institution: Technical University of Munich

Language: English

Topic: Hand Gesture Recognition Systems

Subfield: Human-Computer Interaction

Field: Computer Science

Domain: Physical Sciences

SDGs: Reduced inequalities, Quality education

Open Access status: gold

Estimated APC paid: $1,207