How to Stop Prompt Injection in AI Agents

What prompt injection is

A language model takes in one stream of text: your system prompt, the user's message, and anything else you add, such as documents or tool results. It has no reliable way to tell which parts are your instructions and which are just data. So if some of that data says "ignore your previous instructions and do this instead," the model may do it.

There are two kinds:

• Direct injection: the user types the attack into the chat. You can at least see who sent it.
• Indirect injection: the attack is hidden in content the agent reads while doing its job: a web page, an email, a PDF, a support ticket, a code comment, or the output of another tool. The user may never see it. This is the bigger risk for agents.

Prompt injection is number one on the OWASP Top 10 for LLM Applications, and it's still an open problem. No model today is immune.

Why agents make it much worse

When a chatbot is injected, the worst case is usually that it says something it shouldn't. When an agent is injected, the attacker gets to use the agent's tools and permissions. If the agent can issue refunds, the attacker can issue refunds. If it can read your CRM and send email, the attacker can email your customer list to themselves.

So the real question isn't "can our agent be injected?" Assume it can. The question is "what's the worst thing it can do when it is?" Most of the defenses below are about making that answer boring.

A realistic example

Notice what would have stopped it: the refund going to an order the sender doesn't own, a confirmation email to an outside domain, or a human approving the refund. None of those depend on the model noticing the trick.

What doesn't work on its own

• Asking the model to ignore malicious instructions. Worth adding to your prompt, but attackers rephrase, role-play, split instructions across messages, and use other languages. It lowers the hit rate. It doesn't close the hole.
• Keyword filters. Blocking phrases like "ignore previous instructions" catches the laziest attacks and misses everything else.
• A second AI as the only guard. Using another model to spot injections helps, but that model can be fooled too. Fine as one layer, risky as the only one.

Seven defenses that do work

1. Give the agent less power

The most effective defense is also the least exciting. Remove tools the agent doesn't need, narrow the permissions of the ones it does, and split "read" tools from "do" tools. An agent that can only look up orders can't be tricked into refunding them. Our AI agent security checklist walks through this in detail.

2. Label outside content clearly

Wrap untrusted content in clear markers, tell the model that text inside them is data and never instructions, and keep your real instructions in the system prompt. This doesn't make injection impossible, but it makes it noticeably harder.

3. Check every tool call in code

Before your code runs a tool, check the request against rules the model can't change. Is this tool allowed for this user? Does the order belong to this customer? Is the amount under the limit? Is the destination on the allowlist? Here's the idea in a few lines of Python:

ALLOWED_EMAIL_DOMAINS = {"yourcompany.com"}
MAX_REFUND = 500

def check_tool_call(user, tool, args):
    if tool == "issue_refund":
        order = get_order(args["order_id"])
        if order.customer_id != user.customer_id:
            raise Blocked("Order doesn't belong to this customer")
        if args["amount"] > min(order.total, MAX_REFUND):
            raise Blocked("Refund over limit, needs a human")
    if tool == "send_email":
        domain = args["to"].rsplit("@", 1)[-1].lower()
        if domain not in ALLOWED_EMAIL_DOMAINS:
            raise Blocked("Outside recipient, needs a human")
    return True

The model can be fooled into asking. It can't be fooled into passing this check.

4. Require approval for high-risk actions, and enforce it

For actions you can't undo, a person should approve the exact action before it runs. The approval has to be checked by the system that carries out the action, tied to that specific action and amount, and usable only once. Otherwise an injected agent, or an attacker who captured a request, can skip or replay it.

This is the job A2SPA was built for. Each high-risk action must carry a valid signature for that exact request, and the protected system blocks anything that's unsigned, changed, expired, or replayed. A fooled model can ask for anything. Without a valid signature, nothing happens.

5. Watch what goes out, not just what comes in

Attackers often aim to get data out. Watch for an agent putting secrets, personal data, or internal links into replies, URLs, or image links (a markdown image pointing to an outside server can leak data just by loading). Strip or block links to unknown domains in agent output.

6. Keep secrets out of the context

If an API key or internal detail is in the prompt, an injection can get the agent to repeat it. Keep credentials inside your tool code, where the model can't see them.

7. Log everything and set alerts

Record every tool call with its arguments and whether it was allowed or blocked. A sudden run of blocked calls, or calls to tools a user doesn't normally trigger, is often the first sign of an attack. You can't respond to what you can't see.

How to test your agent

Before launch, and after any change to tools or data sources, try these:

• Hide an instruction in a document, web page, or email the agent will read, and see if it acts on it.
• Ask the agent to reveal its system prompt or any keys it knows.
• Ask it to act on another user's account or order.
• Ask for an action just over each limit (amount, recipients, count).
• Ask it to send data to an outside email address or URL.
• Replay an approved request and confirm the second attempt is blocked.

If any of these work, the fix is usually one of the seven defenses above, most often a missing check in code (defense 3) or an approval the system doesn't actually enforce (defense 4).

Common questions