Work

A unified multimodal learning framework for stress detection using IoT biomarkers and NLP with return-aligned decision transformers

Year: 2026

Type: article

Abstract: Early mental stress detection is important for timely intervention and proactive mental healthcare. However, unimodal stress detection methods often fail to capture the complexity of human stress responses, since physiological and linguistic signals represent complementary yet heterogeneous manifestations of affective state. To address this limitation, this study proposes UML-RADT-WOA, a unified multimodal framework for binary stress detection using wearable physiological signals from WESAD and stress-related textual posts from Dreaddit. The framework integrates MMADN-based preprocessing for physiology, DistilBERT-based textual representation, and class-conditioned Unified Multimodal Learning pairing for controlled multimodal fusion. A Modality-Token Return-Aligned Decision Transformer projects both modalities into a shared latent space, while Gated Multimodal Fusion suppresses modality-specific noise before final classification. Hyperparameters are optimized using the Wombat Optimization Algorithm. Under 15-fold Leave-One-Subject-Out cross-validation, the framework achieves 84.52% accuracy, 84.11% macro-F1, 89.52% ROC-AUC, and 85.88% AUPRC. The results indicate that the proposed framework offers an effective and reproducible IoT-enabled solution for multimodal stress detection.

Source: Discover Internet of Things

DOI: 10.1007/s43926-026-00480-x