Enhancing pilot cognitive resilience through neuroergonomic-informed adaptive human-machine interaction for spatial disorientation mitigation
Type: dissertation
Abstract: Spatial Disorientation (SD) remains a critical threat to aviation safety, contributing to a significant percentage of fatal accidents. Existing SD mitigation strategies are often static and fail to adequately address the neurocognitive underpinnings of SD, such as Loss of Attention Control (LoAC) and a breakdown in cognitive resilience. Current approaches, including instrument training and auditory alerts, often fall short in dynamically adapting to pilots' evolving cognitive states. This thesis addresses this critical gap by establishing a novel neuro-ergonomic framework designed to enhance pilot cognitive resilience through adaptive Human-Machine Interaction (HMI). The research unfolds across three interconnected stages. First, a comprehensive neuro-ergonomic investigation employs electroencephalography (EEG) and eye-tracking to decode pilots' neural and ocular signatures across four distinct SD types (No SD, Type I: unrecognised; Type II: recognised; Type III: incapacitating) during simulated flight phases. Based on these defined signatures, advanced machine learning models were developed and applied to classify a pilot's cognitive state and predict SD onset with an accuracy of 96.8% up to 20 seconds prior to symptom manifestation. Second, to further refine predictive capabilities, we developed Hidden Markov Models (HMMs) to identify phase-specific scanning patterns, including Areas of Interest (AOI) transitions and scanning sequence strategies. Additionally, a hybrid approach combining Conditional Random Fields (CRF) with LightGBM, utilizing a sliding window technique, enabled fine-grained, real-time prediction of visual scanning anomalies. Leveraging these predictions of SD occurrence and visual scanning anomalies, the Adaptive Attention-Awareness Pilot-Aircraft Interaction System (3A-PAI) was proposed. The successful integration of these models into a closed-loop HMI provides a concrete solution, offering a new HMI framework for pilot support by shifting the focus from reactive assistance to proactive, real-time intervention. Third, to explore effective interaction paradigms, this phase investigated how enhanced display technologies could naturally improve pilot cognitive resilience, especially SD. The emergence of technologies like Head-Up Displays (HUDs) offers promising opportunities to revolutionise the cockpit experience by providing pilots with more intuitive and contextualised information. Two enhanced HUD prototypes—a holographic visor and a projection-based system—were engineered to optimise the future pilot-aircraft interaction paradigm within the cockpit. Among these, AR-glasses based HUD could broad the interaction boundary of cockpit, providing multimodal interaction. The projection-based HUD demonstrated more natural interaction approach and a superior user experience with lower learning costs. Finally, a closed-loop collaborative HMI system for SD mitigation was prototyped, where real-time SD predictions trigger context-aware HUD interventions to provide adaptive support to pilots. This work collectively presents a novel, neuro-ergonomic framework that not only advances the understanding of four distinct SD types but also delivers a pragmatic, adaptive HMI solution, thereby setting a new trajectory for pilot-centric cockpit design and significantly enhancing aviation safety.
Author: Yuan, Xin
Language: English
Subfield: Human Factors and Ergonomics
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7218291030
