Who Speaks for Whom: Prompt Injection and Authority in an Email-Drafting Agent
Year: 2026
Type: preprint
Abstract: AI systems now act for people: they draft, reply, book and pay. That makes authority a practical question, not an abstract one: who may direct the system, over what, shown how, and for how long. We give a four-part working definition (source, scope, proof, revocation) and illustrate how each part fails. We then test one failure in our own product. Orchestrate drafts email replies for small businesses, and an owner approves each draft before it is sent. We hid 12 manipulative instructions in prospect replies and ran each 3 times on three models, including the production model. Two graders from different providers agreed on 91% of drafts (Cohen's κ = 0.87). By their joint judgement, the production model obeyed the attacker in 14 of 24 drafts (58%; 95% CI 39–76%); the provider's content filter blocked the other 12 requests. A hand check by the author on 60 drafts agreed with the graders on 93% of clearly harmful attacks (κ = 0.78) but only 63% of ambiguous ones. We deployed two changes: the prospect's words were fenced off as untrusted, and a check outside the model held any draft stating facts absent from the business's records. A third change held drafts that claim actions never taken. After the fix the production model still followed the attacker in 11 of 30 and then 6 of 30 drafts across two runs (37% and 20%), Gemini in 1 of 36 and Claude in none. The checks held every draft that both graders judged compliant before it reached the owner as a ready answer. Three drafts flagged by only one grader were not held; all three asked before acting. Fencing cut compliance sharply for Gemini and Claude but not clearly for the production model. The checks outside the model, not fencing, are what kept compliant drafts from reaching owners as ready answers. YaktAI Lab Report 1. Preprint, not peer-reviewed. A follow-up, Report 2, tests the same check against reworded attacks.
Author: Sakhawat, Muhammad
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7234402094
