Work

Towards Safe AI: A Compositional Construction of Safe-visor Architecture for Interconnected Systems

Year: 2025

Type: article

Abstract: This paper focuses on ensuring the safety of interconnected systems that utilize AI-based controllers, which have not been formally verified, for executing complex tasks. The approach involves encapsulating the AI-based controller within a Safe-visor architecture, which includes a Supervisor to monitor inputs from the unverified controller and a Safety Advisor to provide fallback mechanisms that maintain the system’s overall safety. For systems comprising multiple subsystems, the Safe-visor is constructed in a compositional manner. This involves designing the Safety Advisor and Supervisor for each subsystem individually and then integrating them using assume-guarantee reasoning. In developing Safe-visors for subsystems, the notion of hybrid controlled invariant sets is employed, with safety properties defined as linear temporal logic formulas over finite traces. The effectiveness of this approach is demonstrated through a case study involving a vehicular platoon.

Source: IFAC-PapersOnLine

DOI: 10.1016/j.ifacol.2025.07.068

Author: Bingzhuo Zhong

Institution: The Hong Kong University of Science and Technology (Guangzhou)

Language: English

Topic: Formal Methods in Verification

Subfield: Computational Theory and Mathematics

Field: Computer Science

Domain: Physical Sciences

Open Access status: diamond

Funders: Hong Kong University of Science and Technology, Bureau of Education of Guangzhou Municipality

Awards: G5332081803, G8578121900