Physics-Refracted Interdomain Shift Modeling for Cross-Sensor Pilot Action Recognition
Year: 2026
Type: article
Abstract: Monitoring pilot manipulation behavior is central to civil aviation safety. However, calibrated multi-camera rigs cannot be installed in a cockpit, and a single deployable depth camera yields degraded skeletons. Recognition must therefore transfer across a spatiotemporal shift dominated by a change in sensing modality. This paper proposes PRISM (Physics-Refracted Interdomain Shift Modeling). PRISM combines a cascade of degradation operators with alignment layers and treats the domain shift as separable rather than opaque. Five physics-based degradation operators are cascaded at the input, reproducing the principal error mechanisms of a depth camera. Their parameters are estimated from paired recordings rather than assumed. Two feature-space layers then align marginal distributions and class-conditional prototypes. Averaged over five seeds, combining the cascade with the two alignment layers raises the target accuracy of a spatiotemporal convolutional neural network backbone from 52.85% to 98.98% without target labels. PRISM also removes the grid search over alignment strengths. A grid search over 25 strength combinations placed its optimum at the strength combination that PRISM selects. The gain was significant on all six backbones tested, and a leave-one-subject-out rotation over the four pilots gave 98.91 ± 0.41% accuracy. The framework offers a deployable route to single-sensor cockpit monitoring.
Source: Aerospace
Authors: Yu Shen, Gang Xiao, Gaoqiang Wang, Laizhen Lu, Liu Zeng
Institution: Shanghai Jiao Tong University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,896
OpenAlex ID: W7222960890