Year: 2025
Type: conference-paper
Abstract: AI risk evaluations are key to building safety cases for new AI systems. These evaluations help us track how quickly new AI systems are gaining competencies and propensities to engage in harmful behavior. However, it is not currently clear how well we are doing in covering the surface of potential risks that AI systems could present as they continue to improve. This paper proposes a modelling framework for predicting lags in AI risk evaluations, illustrating our approach using data from METR's evaluations of the ability for AI to complete long tasks. We find in our simulations that a business-as-usual approach to tracking AI risks is vulnerable to substantial detection lags when facing budget constraints. We advise greater adoption of the logistic estimator proposed by METR where appropriate, as it mitigates against such lags.
Author: Paolo Bova
Institution: Teesside University
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: bronze
OpenAlex ID: W4415230361
FWCI:
Cites:
Cited by:
Related to: