A Duty Without an Owner: Indirect Prompt Injection and the Provider–Deployer Gap Under EU Law
Year: 2026
Type: dissertation
Abstract: AI assistants such as ChatGPT, Claude, and Gemini, when connected to a company's own email, documents, or internal tools, can be induced to quietly leak that organisation's data to a stranger instead, not because the underlying model has malfunctioned, but because a hidden instruction buried inside content it was asked to process directed it to do so. This is indirect prompt injection, and the underlying flaw is not limited to chatbots: it affects any AI system a business builds on a large language model that reads content it was not given directly by its own user and can then act on it, whether that is a chat assistant answering a question, a coding agent executing commands on its own, or a backend system with no chat window that a person ever sees. It exists because these models have no structural way to distinguish an instruction from the data they are asked to process, a property, not a design flaw, but the very thing that makes them useful in the first place. A closely related incident at Microsoft 365 Copilot in 2025 illustrates how such an attack can operate in practice: a single crafted email was reported to enable attackers to exfiltrate user data without a click or any action required of the victim. This thesis asks why EU law leaves this gap open, and why no single actor currently has a legal duty to close it. The EU AI Act requires providers of general-purpose AI models to test for this kind of manipulation under Article 55 before releasing a model, and the Code of Practice implementing that duty has already been adopted by the major providers. But that testing happens on the model alone, in isolation, before deployment, and it cannot reach what actually creates the risk in practice: which documents a company connects to its AI, what tools it lets that AI use, and what data a compromised tool call could ultimately reach. Those choices are made separately by every business that deploys the technology, not by the provider, and no current instrument of EU law requires anyone to test them. The AI Act's deployer-facing provisions, Articles 15 and 26, would in principle reach this problem, but they apply only where a system is independently classified as high-risk, which the ordinary enterprise deployment this thesis examines is not, and the 2026 Digital Omnibus reform has pushed even that narrower obligation back to December 2027 at the earliest. The GDPR applies throughout this period in principle, but Article 32 provides no benchmark for what constitutes adequate protection against this specific and relatively new risk, leaving both businesses and supervisory authorities without settled guidance on what compliance requires. Comparable regimes elsewhere in EU law, including the operational resilience testing required under the Digital Operational Resilience Act (DORA) and the cybersecurity duties imposed by the second Network and Information Security Directive (NIS2), show that deployment-specific testing obligations of exactly this kind are neither novel nor unworkable in practice; they simply have not yet been extended to this particular risk. Having identified this provider-deployer gap, the thesis proposes two measures to narrow it without waiting for legislative reform or judicial development that would fully close it. In the short term, it drafts a Data Processing Agreement clause that businesses can adopt immediately, allocating testing and incident-response responsibilities between themselves and their AI provider. In the longer term, it proposes a statutory disclosure duty requiring providers to publish exactly what their pre-market testing did, and did not, cover, giving every deploying business, not only those with the bargaining power to negotiate favourable contract terms, the same baseline information about the risk they are taking on.
Language: Italian
Subfield: Law
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7221381340