Work

Key Instance-Based Spatio-Temporal Network for Group Activity Recognition

Year: 2026

Type: article

Abstract: Group activity recognition involves detecting the collective actions performed by a group of individuals, where identifying the key actors and key frames is crucial for understanding the group’s behavior. To tackle this challenge, we propose a spatio-temporal reasoning framework that leverages key instances. Our key instance identification module effectively detects key roles and frames from video sequences, while a graph-based reasoning model dynamically aggregates the features of related actors. We extract joint features and RGB features from video sequences, and these are fused using our multi-modal fusion TCT module, which improves the representation power of the original features. To better understand group activity through spatio-temporal correlations, we further utilize an enhanced cross-transformer module for spatio-temporal synchronous reasoning, considering both time and space dimensions. Our method has been tested on two public datasets, showing that it achieves high accuracy and surpasses many state-of-the-art approaches.

Source: ACM Transactions on Multimedia Computing Communications and Applications

DOI: 10.1145/3805798

Authors: Yaochen Li, Haoting He, Yutong Wang, Gaojie Li, Junyi Wu, Yuehu Liu

Institution: Xi'an Jiaotong University

Language: English

Topic: Human Pose and Action Recognition

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: National Natural Science Foundation of China

Award: G2024157573