Get started
Work

Analysis of the AI prompt used to improve safety management system accessibility

Year: 2025
Type: article
Abstract: This paper presents a timely and highly relevant investigation for evaluating the effectiveness of AI prompt engineering in improving the safety management system. The study focuses on upstream oil and gas operations, which frequently struggle with quick access to essential safety information. The study investigated an internal AI system configured to access company health, safety, and environmental documentation repositories. The performance is assessed using 30 safety-related prompts across six distinct operational domains, each evaluated on the criteria of relevance, clarity, and completeness, with scores ranging from 1 to 5. Results revealed a trimodal performance distribution, with 60% perfect scores, 30% complete failures attributed to indexing and contextual interpretation issues, and 10% moderate performance( characterized by partial correctness or some degree of inaccuracy). A particularly insightful observation was that high clarity scores were sometimes paradoxically paired with incorrect citations, suggesting that the AI could convincingly present erroneous information. From a process safety perspective, this paper discusses the significant potential of AI to mitigate human error stemming from information scarcity in critical operations. It also critically examines the inherent risks of AI-generated inaccuracies that could lead to potential events in safety-critical contexts. The study recommends the adoption of a hybrid human-AI system, rigorous technical transparency, expanded evaluation methodologies, and a robust framework for addressing ethical considerations to ensure the responsible and effective deployment of AI in hazardous environments.
DOI: 10.1016/j.jlp.2025.105834
Language: English
Open Access status: hybrid
Estimated APC paid: $4,020
OpenAlex ID: W4415681393
FWCI: 1.763
Cites: 12
Cited by: 2
Related to: