WiFi-Guided Multimodal Sensing Using Adaptive Modality Decision With Cost Priority
Year: 2026
Type: article
Abstract: Non-contact human activity recognition (HAR) is a fundamental capability in pervasive computing systems, enabling applications such as health monitoring and smart homes. Although recent multimodal approaches that integrate WiFi and video improve recognition performance, they typically require continuous processing of all modalities. This processing paradigm results in substantial computational overhead and limits their practicality in real-time, resource-constrained environments. To address this limitation, we propose CoMS, a cost-prioritized adaptive multimodal sensing approach to effectively balance the accuracy-efficiency trade-off for contactless HAR. Instead of always activating all modalities, CoMS learns to dynamically determine whether low-cost WiFi sensing is sufficient or whether high-cost visual sensing should be invoked. We introduce a suppressive modality decision policy that explicitly constrains the use of high-cost modalities during learning, encouraging CoMS to rely on the lightweight sensing modality whenever possible. Meanwhile, CoMS integrates modality decision with activity recognition through a differentiable mechanism. This joint optimization scheme allows sensing decisions to be guided by the recognition objective rather than optimized independently, thereby preserving discriminative capability under constrained modality usage. Extensive experiments on self-collected and public datasets demonstrate that CoMS achieves a recognition accuracy of up to 97.54%, while delivering up to$5\times$faster endto- end inference than representative multimodal HAR methods. These results highlight the effectiveness of CoMS in providing accurate, efficient, and practical HAR for real-world non-contact sensing scenarios.
Language: English
Subfield: Electrical and Electronic Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $2,645
OpenAlex ID: W7224464906