Estudio sobre modelos de aprendizaje profundo basados en poses para Traducción de Lengua de Señas sin glosas
Year: 2024
Type: article
Abstract: Sign Language Translation (SLT) is a challenging taskdue to its cross-domain nature, different grammarsand lack of data. Currently, many SLT models relyon intermediate gloss annotations as outputs or latentpriors. Glosses can help models to correctly segmentand align signs to better understand the video. How-ever, the use of glosses comes with significant limi-tations, since obtaining annotations is quite difficult.Therefore, scaling gloss-based models to millions ofsamples remains impractical, specially considering thescarcity of sign language datasets. In a similar fashion,many models use video data that requires larger mod-els which typically only work on high end GPUs, andare less invariant to signers appearance and context.In this work we propose a gloss-free pose-based SLTmodel. Using the extracted pose as feature allow fora sign significant reduction in the dimensionality ofthe data and the size of the model. We evaluate thestate of the art, compare available models and developa keypoint-based Transformer model for gloss-freeSLT, trained on RWTH-Phoenix, a standard dataset forbenchmarking SLT models alongside GSL, a simplerlaboratory-made Greek Sign Language dataset.
Institution: Universidad Nacional de La Plata
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7009744420
FWCI:
Cites:
Cited by:
Related to: