A Certificate of Release to Service for AI Agents: Pilot 0, Testing AI Models as Advisory Reviewers of Finished Work Packs
Year: 2026
Type: preprint
Abstract: A Certificate of Release to Service is signed in aircraft maintenance only after checking that a work pack's records support the claim that work is finished. This pilot asks whether an AI model, acting as an advisory reviewer that never signs, can notice when a finished pack's evidence does not support its "done" statements. Eight tasks (five invented aviation style tasks, three derived from one software project) each had a faulty pack with one seeded fault and a clean twin. Three models reviewed each pack twice, scored by code. Every model caught, located and exactly quoted 16 of 16 seeded faults (Wilson 95% lower bound 0.81), with no invented quotes. False alarms on clean twins were 6 to 8 of 16 in run 1 and 3 to 6 of 16 after a post hoc repair of four twins. The pilot is small, self registered and not validated for real maintenance.
Author: Adekunle, Isaac
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225809496
Cites:
Cited by:
Related to: