Work

SPIDMA: skeletal-posture-based human intrusion detection with multimodal MR alerts for mechanical hazard zone

Year: 2026

Type: article

Abstract: In industrial environments, human intrusion into mechanical hazard zone is a primary cause of occupational injuries. Accurately identifying such intrusions and implementing proactive warnings in complex scenarios remains a critical challenge in safety monitoring. This study proposes a proactive warning framework combining skeletal keypoint recognition and multimodal mixed reality (MR), named SPIDMA. This method extracts human skeletal keypoints from surveillance videos for posture analysis, dynamically adjusts the keypoint set used for intrusion judgment, and achieves high-precision detection utilizing a directed cross-product sign-flip logic relative to the warning line. To overcome the limitations of traditional alarms in visual blind spots and noisy environments, the system introduces a warning mechanism based on a mixed-reality head-mounted display (MR-HMD). Upon detecting an intrusion, it instantly delivers a multimodal alert integrating holographic visuals and spatial audio to operators, enabling immediate hazard avoidance. Experimental results demonstrate that SPIDMA achieves an accuracy, precision, recall, and F1-score of 89.52%, 88.61%, 90.00%, and 89.30%, respectively. Compared with the YOLO bounding-box center baseline, its accuracy and precision are relatively improved by 50.9% and 62.8%, respectively, significantly reducing false positives while maintaining high recall. Additionally, the end-to-end system latency is controlled between 211.6 ms and 286.4 ms, satisfying real-time operational requirements. SPIDMA enhances detection accuracy under complex postures and provides an instant warning response, offering a novel paradigm for proactive safety in industrial collaborative workspaces.

Source: Virtual Reality

DOI: 10.1007/s10055-026-01472-6

Authors: Yukang Hou, Jian Lv, Qingsheng Xie, Ning Zhang, Jian Mu, Bo Lin

Institution: Guizhou University

Language: English

Topic: Human Pose and Action Recognition

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $2,990