Work

A multi-guideline clinical agent for vascular surgery: architecture, implementation, and benchmark evaluation

Year: 2026

Type: article

Abstract: BACKGROUND: The clinical practice guidelines (CPGs) published by the European Society for Vascular Surgery (ESVS) span approximately 1100 pages whose simultaneous cross-referencing is impractical at the point of care. Unguarded large language models (LLMs) carry hallucination risk, motivating a guideline-locked design. METHODS: We describe the architecture and development of a context-aware clinical agent encompassing the full ESVS corpus of 14 CPGs. The system extends a validated single-guideline GraphRAG prototype with an LLM-driven Guideline Router, an embedded Context Gate that withholds retrieval when decision-critical parameters are absent, dual-channel retrieval, and GPT-5 synthesis under a mandatory citation template. An automated benchmark assessed the final system on 55 vignettes, scored by 2 blinded LLM judges over 3 runs (330 judgments), with clinician adjudication of every flagged claim. RESULTS: The system stayed within its locked corpus, declined all out-of-scope queries, and achieved 98.4% guideline-routing recall. The Context Gate showed 91.9% specificity and 78.6% sensitivity for underspecified queries, and all 259 cited recommendations were verifiable within the locked corpus. Automated judges flagged an unsupported claim in 18.2% of items under a sensitive screening definition; on adjudication by 2 vascular surgeons, 7.3% of items contained a clinically consequential error, none graded capable of causing harm. Clinical acceptability of complete answers was not scored and is reserved for prospective expert evaluation. CONCLUSION: These results support the feasibility of a multi-guideline context-aware clinical agent for vascular surgery; citation traceability and locked-corpus provenance facilitate future auditability.

Source: Journal of the American Medical Informatics Association

DOI: 10.1093/jamia/ocag178

Authors: Vangelis G Alexiou, Areti Vassiliou, Bauer E Sumpio, George Geroulakos, Stavros K Kakkos

Institutions: University of Patras, Alfa Institute of Biomedical Sciences, University General Hospital Attikon, Yale University, National and Kapodistrian University of Athens

Language: English

Topic: Artificial Intelligence in Healthcare and Education

Subfield: Health Informatics

Field: Medicine

Domain: Health Sciences

SDG: Good health and well-being

Open Access status: hybrid

Estimated APC paid: $4,332