DRIVE‐SAFE: Data‐Driven Robustness and Informed Validation for Evolving Specifications via Formal Evaluation
Year: 2026
Type: article
Abstract: We present a regulator‐driven framework for the temporal evaluation of learning‐based, black‐box autonomous robot policies under evolving safety requirements. Motivated by real‐world certification processes in which regulators often assess observable behavior in inaccessible black‐box policies, we simulate a scenario where deployed policies must maintain ongoing compliance as human‐defined safety rules change over time. Natural‐language safety requirements are translated into Signal Temporal Logic (STL) specifications, which are then used to quantitatively evaluate rollout traces from black‐box policies. We compute Total Robustness Value (TRV), Largest Robustness Value (LRV), and Average Violation Robustness Voting (AVRV), to summarize average performance, worst‐case violation, and average specification violation across a set of rollout trajectories. These metrics are combined with domain‐informed importance weights to provide targeted feedback to model designers, indicating where retraining or redesign is most needed. In a virtual driving scenario, iterative retraining increased satisfaction of a global speed limit specification by 72% and a slippage mitigation specification by 32%. When the speed‐limit requirement was tightened, the resulting retrained model increased satisfaction by 47% but revealed trade‐offs with competing safety objectives. Overall, our approach integrates temporal‐logic‐based robustness metrics with statistically meaningful evaluation, bridging regulator oversight and model improvement under evolving safety requirements.
Source: Advanced Robotics Research
Authors: Kristy Sakano, Jianyu An, Dinesh Manocha, Huan Xu
Institution: University of Maryland, College Park
Language: English
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $2,610
OpenAlex ID: W7166308335