Work

A Multi-Mode Deep Learning Framework for Suspicious Behavior Detection in Controlled Environments

Year: 2026
Type: article
Abstract: Surveillance in controlled environments such as military bases and other restricted facilities requires timely detection of suspicious activities. This paper presents a real-time multi-mode deep learning framework that integrates YOLOv8-based person detection, Deep SORT multi-object tracking, spatio-temporal feature extraction, and behavior recognition through rule-based, supervised, and self-learning operating modes. A six-mode ablation study compares hybrid and no-fallback variants under a shared deployment pipeline. Tracking evaluation on the PETS09 surveillance dataset confirms stable person trajectories suitable for downstream analysis, and a two-stage sensitivity study identifies operational thresholds for loitering duration, radius, and running speed. Under independent manual ground-truth validation on a controlled deployment video, the self-learning hybrid mode achieved the highest accuracy and macro F1 among the six evaluated modes, while supervised-synthetic and rule-based modes remain competitive. Removing the rule-based fallback causes both LSTM-only ablations to degrade toward near-random behavior, demonstrating that fallback fusion is functionally essential. Self-learning achieves this performance without any manually labeled deployment data, suggesting potential practical applicability for controlled surveillance environments, pending further validation. Preliminary cross-scene evaluation on a second deployment video shows that all hybrid modes maintain high agreement with a rule-based reference and reproduce the same no-fallback degradation pattern across scenes. On the manually annotated deployment subset, the self-learning hybrid mode attains an accuracy of 0.899 and a macro F1 of 0.713, while both no-fallback ablations collapse to near-random accuracy of approximately 0.50, a 38-40 percentage-point accuracy drop across the two ablations relative to their fallback-enabled counterparts, indicating that rule-based fallback fusion is necessary for maintaining above-chance performance in this framework.
Language: English
Open Access status: hybrid
Estimated APC paid:
OpenAlex ID: W7236748989
FWCI:
Cites: 32
Cited by:
Related to: 10