Deep functional multiple index models with an application to speech emotion recognition
Year: 2025
Type: article
Abstract: Speech Emotion Recognition (SER) plays a crucial role in advancing human–computer interaction, speech processing capabilities and making synthesized voice sound more natural. This paper presents a novel deep learning architecture tailored for the multiple-index functional model, with a focus on advancing SER. This architecture uniquely integrates adaptive basis layers and an automated data transformation search, using transformer-encoder layers, within the deep learning framework. Simulations show promising performance in both regression and classification tasks, offering a valuable new tool for the functional data analysis community. While chunk-level emotion recognition presents a significant challenge due to the dynamic and subtle nature of speech, framing the problem with functional data provides a powerful way to capture these temporal dependencies and improve performance. By representing Mel Frequency Cepstral Coefficients (MFCCs) as a time-varying multivariate functional object, we derive features specifically for chunk-level SER. Applying our model to two benchmark databases (IEMOCAP and SAVEE), we achieve an accuracy of 59.18% on IEMOCAP and 78.75% on SAVEE, demonstrating its effectiveness in analyzing speech data for emotional content. This result is particularly relevant to the SER community as it highlights the potential of utilizing MFCCs within a functional data framework for emotion recognition. This work provides two significant contributions: a novel solution for multiple-index functional model prediction with broad applicability, and a new methodological approach to SER that advances the field by leveraging the power of functional data analysis.
Source: Array
Institution: Institut Supérieur de l'Électronique et du Numérique
Language: English
Topic: Emotion and Mood Recognition
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $1,680
OpenAlex ID: W4416980373