Work

Behavioural Drift in Model-Checker-Accepted Repairs: A Measurement and Witness Audit

Year: 2026
Type: preprint
Abstract: A model checker can accept a memory-safety repair even when the repaired program produces a different result. We re-evaluated previously accepted FormAI repairs on held-out inputs that passed the original-program safety screens and met a mechanical test of input well-formedness. Observable behaviour changed for 24/86 repairs (27.9%; program-cluster 95% interval 15.1–42.4%), across 12 of 29 eligible programs. Replaying the tests without an output limit preserved this count. An AI-assisted audit classified 2/86 repairs as likely regressions in computed output; independent human adjudication was unavailable. Other changes included input rejection and corrections to an erroneous original. On contest programs, the combined corpus exposed changes in 63/112 repairs, compared with 0/112 on retained official tests and 0/19 on dataset-generated tests. Some inputs that met the mechanical rule fell outside the problem domain. These results show how strongly the estimated rate of behavioural change depends on the test inputs and the evidence available for judging the original output.
Language: English
Subfield: Software
Open Access status: green
OpenAlex ID: W7236706771
Cites:
Cited by:
Related to: 10