Year: 2026
Type: article
Abstract: Clinical AI agents combine prediction, retrieval, tool use, and workflow action, creating safety risks that model-level evaluation alone does not capture. This article derives a responsible architecture through a structured narrative evidence synthesis and examines its control logic in an end-to-end sepsis-deterioration scenario. The synthesis yielded three cross-cutting requirements: separation of orchestration from bounded execution, risk-defined human approval and override, and continuous audit and post-deployment monitoring. In the sepsis walkthrough, the architecture blocked unauthorized actions, suppressed recommendations when inputs or evidence were insufficient, escalated an unacknowledged high-risk alert, and preserved the event trail required to reconstruct the decision. The scenario verifies controllability and auditability at design level. Prospective silent and early-stage clinical evaluation remain necessary to measure missed deterioration, inappropriate escalation, unsupported recommendations, clinician override, and time to intervention.
Author: Bajpai , Sachin
Institution: PricewaterhouseCoopers (United States)
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: diamond
OpenAlex ID: W7204338659
FWCI:
Cites:
Cited by:
Related to: