Efficient spatio-temporal modeling for human action recognition from RGB streams using unitary temporal encoding and adaptive consistency refinement
Year: 2026
Type: article
Abstract: The problem of Human Action Recognition (HAR) continues to be difficult with the intricate temporal interactions, superfluous frames, and minor visual variations that usually define similar actions. A large number of current approaches are based on either transformer or multimodal architectures, which are computationally costly and cannot be used in real-time or resource constrained systems. To overcome these shortcomings, we proposed a novel and lightweight HAR model that integrates spatial, temporal and consistency modeling in an efficient architecture. In our model, we combine EfficientNet-B2, used in efficient spatial feature extraction, with Unitary Temporal Encoder (UTE), to train long-range temporal dependencies, and Adaptive Temporal Consistency Module (ATCM), to improve local temporal consistency. The proposed system is trained and tested on the datasets UCF101 and HMDB51 with a Top-1 accuracy of 97.10% and 87% under RGB input only which is competitive among RGB-based HAR methods while maintaining low computational cost. The 9.3 million parameter model with an inference rate of 9.5 ms per 16-frame video clip is highly accurate and efficient, thus suitable in inference in real-time and at edges. Ablation studies further demonstrate that the proposed components contribute consistent performance improvements across the evaluated benchmarks.
Source: Scientific Reports
Authors: Abdul Majid, Yulin Wang, Aqsa, Zhang Xing, Anwarullah, Melaku Berhe Belay
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,690
OpenAlex ID: W7166691035
FWCI:
Cites:
Cited by:
Related to: