Work

Prompt Injection Resistance in Clinical LLM Agents via Structured Medical Ontology Alignment

Year: 2026
Type: article
Abstract: Large language model agents deployed in clinical settings offer transformative potential for decision support, patient communication, and workflow automation. However, their vulnerability to prompt injection attacks poses a critical safety risk, especially when adversarial inputs can manipulate model outputs to produce harmful or misleading medical advice. This paper proposes a structural defense framework based on the alignment of clinical large language model agents with a formal medical ontology, such as the Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) or the Unified Medical Language System (UMLS). By constraining the agent’s reasoning and generation processes to a structured representation of medical knowledge, the system can detect and reject inputs that deviate from clinically valid pathways. We present an architectural design that integrates ontological grounding at multiple stages of the agent pipeline, including input preprocessing, context injection, and output validation. The approach is evaluated against a taxonomy of prompt injection techniques, including direct, indirect, and multi-turn attacks. Results demonstrate that ontology-aligned agents exhibit significantly higher resistance to adversarial manipulations compared to unconstrained baseline models, while maintaining clinical accuracy and fluency. The paper also discusses the trade-offs between security and expressiveness, the computational overhead of ontology integration, and the implications for regulatory compliance and deployment in resource-constrained healthcare environments. We argue that structured ontology alignment represents a promising direction for building trustworthy clinical large language model agents that can safely operate in adversarial open-world interactions.
Language: English
Open Access status: diamond
OpenAlex ID: W7220193228
FWCI:
Cites: 10
Cited by:
Related to: 9