The Presence of Articulatory Representations in Self-Supervised Speech Models: A Cross-Linguistic Perspective
Year: 2026
Type: dissertation
Abstract: Recent advancements in self-supervised learning (SSL) speech models have demonstrated their ability to capture rich linguistic and acoustic representations from raw audio.However, the extent to which these models encode articulatory information, particularly across typologically diverse languages, remains underexplored.This thesis investigates the presence of articulatory representations in SSL models from a crosslinguistic perspective, focusing on Finnish and Russian, two languages with distinct phonetic and phonological characteristics.Using electromagnetic articulography (EMA) data from bilingual speakers, we evaluate the correlation between latent representations extracted from SSL models (including Wav2Vec 2.0, XLSR-53, and MMS-300m) and actual articulatory movements.Our experiments reveal that SSL models achieve strong articulatory prediction performance (Pearson r up to 0.68), even with minimal training data (∼ 5 minutes), suggesting efficient encoding of articulatory features.Multilingual models, particularly MMS-300m, outperform monolingual ones, indicating that broader language exposure enhances articulatory generalization.Layer-wise analysis shows that intermediate layers best capture articulatory information, while sensor-specific performance varies, with tongue movements being more predictable than lip movements.Additionally, we examine the impact of task type (sentence repetition vs. spontaneous speech) and language proficiency, considering a spectrum from native speakers to varying degrees of second language proficiency, on model performance.Results indicate that structured tasks yield higher accuracy, while models demonstrate strong generalization across different proficiency levels, highlighting their robustness.These findings contribute to the interpretability of SSL models and emphasize their potential applications in speech technology and clinical diagnostics.By bridging computational speech processing and articulatory phonetics, this thesis advances our understanding of how SSL models represent the physical dynamics of speech production across languages.
Author: Pollio San Pedro, Ailín
Language: English
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Keywords: articulatory representation, self-supervised speech models, cross-lingual generalization, electromagnetic articulography, Finnish language, Russian, multilingual speech models, wav2vec 2.0, tongue movement, lip movement, second language proficiency, spontaneous speech, speech production, articulatory phonetics
Open Access status: green
OpenAlex ID: W7215301960
