Work

Safety and Reliability Patterns for Autonomous AI Agents - In MedTech context

Year: 2026
Type: dissertation
Abstract: Autonomous AI agents that plan, call tools and act with little supervision are moving into regulated settings such as medical technology, where a mistaken action is a clinical incident rather than an inconvenience. Recent evaluations show that greater model capability does not bring safer behaviour, so safety cannot rest on the model alone. The literature offers many agent-safety techniques and several broad architectures, yet it does not tell a MedTech team which mechanisms a given system needs, when each becomes mandatory, or how each maps to the regulations the system must satisfy. This thesis responds with three contributions. The first is a safety and reliability pattern framework, Aegis-3, scoped to medical technology: sixteen patterns across architectural, process and implementation levels, indexed by four operational contexts and four action-autonomy tiers, with a decision matrix that turns a deployment classification into a concrete pattern selection, and an explicit mapping from each pattern to the EU AI Act, the EU MDR, and other regulations. The second is a proof-of-concept that instantiates a subset of the framework: a tool-using agent for care planning in multimorbid elderly patients, with deterministic guardrails, a non-bypassable human approval gate, and a tamper-evident audit trail. The third is an evaluation method that measures the safety architecture rather than clinical correctness, through controlled injection of unsafe proposals and a cross-model comparison harness. Across ten synthetic patient profiles the deterministic guardrails blocked every injected unsafe medication proposal while blocking none of the safe controls, and the approval gate admitted no commit without a recorded human decision. These guarantees are independent of the model behind the agent by construction, because the guardrails read structured output rather than model prose, and a self-correction loop kept the pipeline robust to models that break the output contract. The evaluation demonstrates that the framework yields buildable and measurable designs rather than a clinical validation.
Language: English
Open Access status: green
OpenAlex ID: W7225846170
Cites: 27
Cited by:
Related to: 10