Trust-Gated RGB-Depth Fusion and Self-Supervised Spatiotemporal Modelling for Crowd Anomaly Detection
Year: 2026
Type: article
Abstract: . Detecting unusual crowd behaviour in surveillance settings is difficult because abnormal events are unpredictable, and relying on only one type of visual input has clear limitations. To address this, we introduce a multimodal framework that combines RGB and Depth data using a dynamic trust-gated fusion mechanism. This allows the system to adjust the level of trust it places in each input stream based on signal quality and environmental conditions. Separate feature extractors and a hybrid temporal modelling approach preserve the unique strengths of each modality, while a spatiotemporal attention mechanism directs the model’s focus toward moving subjects and reduces interference from static background elements. Instead of requiring detailed definitions of every possible abnormal behaviour, the system is trained in a self-supervised way to learn and reconstruct normal crowd patterns. Any significant deviation from these learned patterns—measured through reconstruction errors across both modalities—is treated as a potential anomaly. Tests on crowd surveillance datasets show that this method is more robust in challenging conditions such as low lighting and complex scenes. It also outperforms traditional fusion techniques and standard autoencoder models, while providing interpretable results by highlighting abnormal regions through attention‑based spatiotemporal maps. Key Words: Multimodal Fusion, RGB‑Depth Surveillance, Abnormal Crowd Behaviour Detection, Self‑Supervised Learning, Spatiotemporal Attention
Authors: Ms.Sindhu T, Dr.KrishnaPriya P
Institution: KG College of Arts and Science
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7130430195
FWCI:
Cites:
Cited by:
Related to: