Large language models are now embedded in customer support, internal search, coding assistants and autonomous agents. Each of these deployments adds an attack surface that traditional application security was not designed for. The most important thing to understand is that an LLM cannot reliably distinguish between instructions from you and instructions hidden in the content it processes.
That is the essence of prompt injection, ranked first in the OWASP Top 10 for LLM Applications. Because no current model is immune to it, the right question is not "how do we block every injection?" but "what can an injected instruction actually achieve in our system?"
1. Map what the model can see and do
Start with an inventory: which data sources can the model read, which tools can it call, with what permissions and on whose behalf? Draw the data flow and mark every point where untrusted content (user input, uploaded documents, emails, web pages, tool outputs) enters the context window. Each of those is an injection entry point.
2. Apply least privilege to the model, not just to users
Treat the LLM as an untrusted component. Give it the narrowest possible permissions, scoped to the current user's own entitlements. A support assistant does not need write access to the CRM; a summarisation feature does not need internet access. If a tool can change data, move money or send messages, it deserves the same scrutiny as a public API.
3. Enforce access control in retrieval
Retrieval-augmented generation (RAG) systems frequently leak data because documents are indexed without their original permissions. Filter retrieval results by the requesting user's access rights before they reach the model. Never rely on the prompt to "only answer with documents the user is allowed to see".
4. Keep humans in the loop for consequential actions
For actions with real-world impact, such as sending emails, executing code, modifying records or making payments, require explicit user confirmation that shows exactly what will happen. Design confirmations so that text generated by the model cannot bypass them.
5. Constrain outputs
Treat model output as untrusted input to the next system. Encode it before rendering to prevent cross-site scripting, validate structured outputs against a schema, and block images or links pointing to unknown domains, a common channel for data exfiltration.
6. Layer guardrails, but don't rely on them alone
Input and output classifiers, system-prompt hardening and content filters raise the cost of attacks and catch a lot of noise. They are useful, but probabilistic. Architecture (permissions, isolation and confirmation) is what limits the impact when guardrails fail.
7. Log, monitor and test continuously
Log prompts, retrieved context, tool calls and responses, with appropriate privacy controls, so you can investigate incidents. Add adversarial test cases to your evaluation suite and run them on every model, prompt or tool change. New jailbreak techniques appear constantly; your tests should evolve with them.
A quick checklist
- Every untrusted input to the model is identified and documented.
- Tools run with the user's permissions, never with a shared super-user account.
- Retrieval enforces document-level access control.
- Consequential actions require human confirmation.
- Outputs are encoded, validated and stripped of untrusted links.
- Security evaluations run automatically before each release.
If you are preparing to launch an LLM feature or an AI agent, an independent AI security test is the fastest way to find out how these controls hold up against a determined attacker.