09.04 · Concept
Defences that survive contact
Rank the real mitigations and say why prompt-level pleading is not one.
Prompt injection is an untrusted-input security problem, not a wording problem. Durable mitigations remove authority, narrow tool capability, validate actions, isolate side effects, and add auditability. Instructions that merely ask the model to ignore hostile text are weak because they share the same channel as the attack.
What this lesson answers
- why prompt hardening does not stop prompt injection
- how to rank mitigations for agent prompt injection
- what defences work against indirect prompt injection
Notes
Prompt injection is not a weird phrasing problem; it is an untrusted-input problem. If your application mixes developer instructions, user text, retrieved documents, web pages, emails, tool outputs, and model reasoning into one context, the model can be persuaded to treat hostile data as instructions. A serious defense starts by assuming some text the model reads is malicious and designing the system so that text cannot directly gain authority.
The strongest mitigations are architectural. Keep privileges narrow, separate trusted instructions from untrusted content where possible, require…
Common questions
- Why is prompt injection treated as an untrusted-input problem?
- The model is reading text from users, documents, tools, web pages, and other sources, and some of that text may be hostile. If that hostile text can influence actions with real authority, the issue is not phrasing. It is the same class of problem as letting untrusted input cross a security boundary.
- What are the strongest defences against indirect prompt injection?
- The strongest defences reduce what the agent is allowed to do. Use narrow privileges, constrained tools, explicit approval for risky operations, output validation before execution, sandboxed side effects, decision logging, and adversarial testing. Filters and guardrails can help, but they are weaker than removing dangerous capability.
- Why is a stronger system prompt not enough?
- A system prompt is still text interpreted by the same model that reads the attacker-controlled content. It can express a policy, but it is not an access-control check, sandbox, permission system, or type boundary. It may stop simple cases, but it should not be trusted to protect secrets or production actions.
Short definition: what is Defences that survive contact?