Work

A dual-branch keypoint Transformer for continuous Korean Sign Language recognition

Year: 2026
Type: article
Abstract: Continuous Korean Sign Language recognition requires practical models that capture manual and non-manual articulation. We propose KDS-Former, a compact dual-branch keypoint Transformer that encodes hand and body–face motion, fuses the temporal representations, and predicts gloss sequences using connectionist temporal classification. On KSL-Guide, KDS-Former-S achieved word error rates (WERs) of \(6.33 \pm 1.68\) % and \(7.61 \pm 2.09\) % under seen-signer/unseen-sentence and unseen-signer/unseen-sentence protocols, respectively, using 2.59 million parameters and 1.91 model-compute GFLOPs. KDS-Former-M achieved \(5.68 \pm 1.98\) % and \(6.27 \pm 2.24\) % using 14.75 million parameters and 9.93 model-compute GFLOPs. The larger systems of Min et al. and TEMPO achieved lower mean WERs across three runs, whereas KDS-Former required fewer parameters, lower model-compute GFLOPs, and lower peak GPU memory usage. Under both primary protocols, the full KDS-Former-S achieved lower mean WER than all five controlled ablation variants, although the unseen-signer/unseen-sentence single-branch result showed substantial between-run variability. Under controlled input corruption, both KDS-Former variants showed smaller mean WER degradation than ST-GCN. These results position KDS-Former as an efficiency-oriented, model-level alternative rather than an accuracy-leading method. Signer-generalization results are limited by evaluation on only two held-out signers, and the reported latency was measured on an RTX 4090 without upstream keypoint extraction, preprocessing, or CTC decoding; broader signer-disjoint and hardware-specific validation is therefore required.
Language: English
Open Access status: gold
Estimated APC paid: $2,690
OpenAlex ID: W7234008813
FWCI:
Cites: 27
Cited by:
Related to: 10