Year: 2026
Type: conference-paper
Abstract: The present research explores existing approaches and methods used to study and evaluate novel system components with increasingly autonomous capabilities, such as AI agents, in safety-critical domains. We report a large-scale review of the human–autonomy teaming (HAT) literature and extend the input-mediator-output model of team effectiveness to guide human-centred assessment, culminating in the IMO-A framework. We then test existing metrics and understand approaches to evaluation used by Human Factors researchers using a novel method which leverages an AIgenerated video generation tool to develop underwater maritime scenarios across levels of autonomy. Our final recommendations serve as a roadmap for progressing HAT evaluation from fragmented, study-specific measurement choices toward a standardised, IMO-A guided, autonomyappropriate and multi-method evidence base that can be translated into practitioner-ready earlyphase HSI protocols for the safe integration of autonomous agents in safety-critical systems.
Institution: University of Nottingham
Language: English
Subfield: Human Factors and Ergonomics
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7161105714