Work

Real-Time Indonesian Sign Language Recognition Using Convolutional Neural Networks and MediaPipe Framework

Year: 2026
Type: article
Abstract: Few hearing Indonesians understand Bahasa Isyarat Indonesia (BISINDO), and most existing recognition systems classify raw images, which makes them sensitive to background, lighting and signer appearance. This study developed a real-time BISINDO alphabet recognition system that extracted 21 three-dimensional landmarks per hand with MediaPipe Hands and classified the normalised landmark vector with a lightweight one-dimensional convolutional neural network (1D-CNN). A dataset of 31,200 samples covering 26 letters was collected from 12 signers, and the model was compared with Random Forest, a multilayer perceptron and an image-based MobileNetV2 under signer-dependent and signer-independent protocols. The 1D-CNN achieved an accuracy of 99.2% in the signer-dependent setting and a macro F1-score of 93.7% on unseen signers, exceeding MobileNetV2 by 12.1 percentage points while using about 40 times fewer parameters. Landmark normalisation contributed the largest improvement. The pipeline ran at about 28 frames per second on a laptop CPU and reached 91.5% accuracy in a live test with five new users, showing that landmark-based recognition was an efficient and robust basis for BISINDO assistive applications.
Language: English
Open Access status: hybrid
OpenAlex ID: W7236574098
FWCI:
Cites:
Cited by:
Related to: 10