Work

Practical reliability-aware temporal state correction for depth-assisted RGB-skeleton action recognition

Year: 2026

Type: article

Abstract: RGB-D action recognition benefits from skeleton motion, RGB appearance, and depth geometry, yet auxiliary modalities are not equally helpful for every sample. We present a reliability-aware temporal state correction method that treats the skeleton representation as the primary temporal state and RGB and local depth features as corrective evidence. A low-cost state-unreliability proxy combines class-normalized prediction entropy with temporal variation between unit-normalized hidden states; both components are dimensionless and bounded before fusion. A monotonic sigmoid gate then controls sample-wise RGB correction, while skeleton-guided local depth features provide a subordinate geometric residual. On NTU RGB+D 60, the full model achieves 93.7%/96.0% under the cross-subject/cross-view protocols; on NTU RGB+D 120, it achieves 90.2%/91.8% under the cross-subject/cross-setup protocols. Under a strictly matched NTU RGB+D 60 cross-subject decomposition, reliability-conditioned RGB correction improves naive integration from 91.8% to 93.1%, and local depth further raises accuracy to 93.7%. The combined proxy obtains an error-detection AUROC of 0.72 on the held-out validation split. We interpret this score only as an empirical conditioning proxy, not as Bayesian or calibrated uncertainty.

Source: Scientific Reports

DOI: 10.1038/s41598-026-74313-w

Authors: Yuanfeng Zhang, Mingyu Zhang, Xiaokun Wang, Shoufeng Tong

Institutions: Changchun University of Science and Technology, Jilin Engineering Normal University, PLA Air Force Aviation University

Language: English

Topic: Human Pose and Action Recognition

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,690

Funder: People's Government of Jilin Province