09.02 · Concept
Private data, untrusted content, and a way out
Identify the three properties that together make exfiltration possible, and break one.
Exfiltration becomes possible when an AI system can see private data, process attacker-controlled content, and send information out through a response or tool. Prompting is not a security boundary. The reliable fix is to remove one of those conditions through tighter context, isolation, filtering, or constrained outputs.
What this lesson answers
- how does prompt injection exfiltrate private data
- why system prompts are not a security boundary
- how to prevent data leaks in AI agents
Notes
Exfiltration through an AI system usually needs three things at the same time: private data the system can access, untrusted content the model is asked to process, and an output or tool path that lets the private data leave. If all three are present, the untrusted content can try to influence the model into revealing secrets, embedding them in a response, calling a tool, or sending them somewhere unintended.
A useful mental model is to treat the model as a confused interpreter sitting between different security zones. Your system prompt, retrieved documents, user messages, web pages, emails,…
Common questions
- What makes prompt injection dangerous in an agent?
- Prompt injection is dangerous when untrusted text can influence a model that also has access to private data and an outbound path. The text may be a web page, email, ticket, document, or tool result. If the agent treats it as instruction rather than data, it can be manipulated into exposing information.
- Can a strong system prompt stop private data exfiltration?
- A strong system prompt helps describe intended behaviour, but it does not isolate secrets. If the model can read both sensitive data and hostile content, the hostile content can keep trying to override the intended behaviour. Security needs architectural controls, not only better wording.
- How do you reduce exfiltration risk in an AI workflow?
- Break one required condition. Keep private data out of context unless it is necessary, separate or filter untrusted content before the model uses it, and restrict what responses or tools can send onwards. Review the feature by tracing what data is reachable, what content is untrusted, and where output can go.