Work

Randomized Modality Mixing with Patchwise RBF Networks for Robust Multimodal Pain Recognition

Year: 2026

Type: article

Abstract: Pain recognition based on multimodal physiological signals remains a challenge, not only because of the limited training data, but also due to the varying responses of individuals. In this article, we present a randomized modality mixing technique (Modmix) for multimodal data augmentation and a patchwise radial basis function (RBF) network designed to improve robustness in limited and highly heterogeneous data. Modmix generates new samples by randomly swapping modalities between existing data points, creating new data in a very simple but effective way. The RBF patch network divides the input into randomly selected, overlapping patches that capture local similarities between modalities. Each patch network is trained end-to-end using stochastic gradient descent. Moreover, the model’s performance is further improved by using multiple independently trained networks and combining them into a single decision. Experiments with the two different pain datasets X-ITE and BioVid were performed under limited training data conditions, where only approximately 30% of the original datasets were used for training. With both datasets the RBF patch network achieved significant improvements for a subset of subjects, resulting in a similar or even slightly better mean accuracy compared to competing related models such as random forest and support vector machine.

Source: Computers

DOI: 10.3390/computers15020127

Authors: Mehmet Erdal, Sascha Gruss, Steffen Walter, Friedhelm Schwenker

Institutions: Universität Ulm, University and Rehabilitation Clinics Ulm

Language: English

Topic: Emotion and Mood Recognition

Subfield: Experimental and Cognitive Psychology

Field: Psychology

Domain: Social Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $2,172