- Home
- AI agents
- Guardrails
- Input and output guardrails
Govern and secure · Guardrails
What are input and output guardrails for AI agents?
Input guardrails check what reaches the model: the user's prompt, retrieved documents, emails, web pages and tool results, scored for prompt attacks, harmful content and sensitive data. Output guardrails check what leaves: the answer and the tool calls the model proposes, for harm, leaked data, ungrounded claims and actions that do not match the request.
In this guide: 83 pages
Key takeaways
- An agent has more than two sides. The platforms now expose four intervention points: user input, tool call, tool response and output. Treat a tool result as input and a proposed tool call as output.
- User prompt attacks and document attacks are different classifiers with different defaults: in Foundry the user prompt attack control is on by default and the document attack control is off until you enable it.
- On Bedrock the prompt attack filter and the PII filter both skip tool call arguments, tool results and tool definitions, so tool traffic needs its own ApplyGuardrail call.
- Rules go on the actions and the exits (tool arguments, domains, amounts); classifiers go on the text. The October 2025 research bypassed 12 published defenses, most at above 90% success, so a classifier alone is never the last line.
What does an input guardrail check?
An input guardrail checks three things before text reaches the model: whether it is an attack, whether it is harmful and whether it carries data that should not go in. The important design choice is to run those checks on every channel the agent reads, not only on the chat box.
Prompt-attack detection comes in two forms. A user prompt attack is the person at the keyboard trying to change the system rules, embed a fake conversation, swap the persona or encode a request to slip past filters. A document attack is an instruction hidden in content the agent fetched: an email, a ticket, a web page, a tool result. Azure Prompt Shields (generally available since August 2024) take the two in separate fields, user prompt and documents, and the document classifier recognises its own categories, such as manipulated content, information gathering, fraud and malware. Bedrock's prompt attack filter covers jailbreaks, prompt injection and, on the Standard tier, prompt leakage. Model Armor detects prompt injection and jailbreaks and returns no match for inputs under three words.
Trust tagging decides what the classifier sees. Bedrock requires input tags around user content when you call InvokeModel; without tags, prompt attacks are not filtered for those calls. Azure Spotlighting (preview, Chat Completions API only) base64-encodes third-party documents so the model treats them as lower trust, at the cost of more tokens and models that mention the encoding in their answers.
Harm categories (hate, sexual, violence, self-harm, plus misconduct on Bedrock and a responsible-AI set on Model Armor), denied topics and word lists complete the input side, together with PII handling: block a prompt with a card number in it, or mask it before the model reads it. Purview DLP for Microsoft 365 Copilot can block prompts containing sensitive information types (preview) and exclude labelled files from grounding.
What does an output guardrail check?
An output guardrail checks the answer for harm, for data that should not leave, for claims the sources do not support and for protected material, and it checks the model's proposed actions against the user's request. Output here means two things: the text the user sees and the tool call the model wants to make.
On text, the same harm and PII classifiers run in the other direction. Bedrock sensitive information filters can mask on output while allowing on input, which suits a summariser that reads customer records and must not repeat them. Protected material detection (generally available for text since August 2024 on Azure; code detection in preview with an index current only to April 2023) catches reproduced lyrics, articles and code.
Grounding checks compare the answer with the sources. Bedrock contextual grounding checks score relevance and grounding against the retrieved context. Automated Reasoning checks (generally available 6 August 2025) go further and test an answer against formal rules extracted from a policy document, in detect mode only, in English (US), with added latency that grows with the policy's variables. Azure groundedness detection is a preview that retires on 30 April 2027 and does not apply to agents, so an Azure agent needs that check in evaluation tooling or in your own code.
On actions, Azure Task Adherence (preview since November 2025) flags a planned tool call that does not match the user's intent, such as a call to clear calendar events when the user asked to view them, and can block it or escalate to a person. The rendering layer belongs here too: what the agent may show as a link or an image is an output decision, and the 2025 leaks through image URLs are why Microsoft blocks markdown image injection deterministically and Salesforce enforced Trusted URLs for Agentforce from 8 September 2025.
Where do tool calls and tool results fit?
A tool result is input the agent did not ask a person for, and a tool call is output that changes the world, so both need checks, and both are where the platform defaults are weakest. The three clouds opened explicit intervention points for them in 2025 and 2026.
Microsoft Foundry's guardrails and controls define four intervention points: user input, tool call (preview, agents only), tool response (preview, agents only) and output. Prompt Shields can scan document attacks at the tool response point for supported tools. The catch is configuration: an agent guardrail fully overrides the model's guardrail, and if the agent guardrail has no control for tool calls, nothing scans them. The troubleshooting section of Microsoft's own page lists "tool call and tool response intervention points not configured" as the first cause of harmful content passing through.
Amazon Bedrock Guardrails evaluate prompts and model responses. The prompt attack filter does not evaluate toolResult content or the tool definitions in toolConfig, and the sensitive information filter does not evaluate tool call arguments, tool results or tool definitions either, so a customer's email address the model passes to a file-writing tool goes through unmasked. Close the gap with a standalone ApplyGuardrail call on tool results before they re-enter the context, and with AgentCore Policy on the tool call itself, which evaluates Cedar rules against the caller's identity and the arguments at the AgentCore Gateway (generally available 3 March 2026).
Google routes tool traffic through Agent Gateway, and Model Armor has screened that traffic since its Agent Gateway integration went generally available on 24 June 2026, after its integration with Google-managed MCP servers on 22 April 2026. Copilot Studio takes a different route: before every tool invocation on a generative-orchestration agent, it can post the prompt, history, prior tool outputs and the proposed call to an external threat detection endpoint (preview), which answers allow or block within one second or the call proceeds by default.
Which checks should be rules, and which should be classifiers?
Put a rule on anything you can name (tools, arguments, domains, amounts, counts) and a classifier on anything you can only describe (an attack, a harmful passage, a name in free text). The table maps each check to the kind that fits, with the limit to remember.
Input and output checks, the kind of control that fits, and its documented limit.
| Check | Side | Kind | Limit to remember |
|---|---|---|---|
| Prompt attack on user text | Input | Classifier | Bypassed by adaptive attacks; trained mainly on English (Prompt Shields) |
| Document attack on fetched content | Input | Classifier plus trust tagging | Off by default in Foundry; Bedrock needs input tags; skips tool results on Bedrock |
| Harm categories, denied topics | Both | Classifier; word lists are rules | False positives; thresholds differ per vendor and language |
| PII and secrets | Both | Classifier with regex rules | Bedrock masking skips tool traffic and invocation logs |
| Grounding | Output | Classifier, or formal check on Bedrock | Azure version retires 30 Apr 2027; Automated Reasoning is detect only and English (US) |
| Tool allowed, arguments in range | Output (tool call) | Rule | Only covers what it was told; needs caller identity in the token |
| Task matches intent | Output (tool call) | Classifier | Task Adherence is preview and may process data in US or EU regions |
| Links, images, email domains | Output | Rule | Allow lists go stale (ForcedLeak used an expired but allow-listed domain) |
A useful test: if a reviewer can write the condition as a sentence with no "looks like" in it, it is a rule. Rules belong on the exits because that is where an injection does damage, and a rule on the exit holds even when the classifier on the entry missed.
How do Microsoft, AWS and Google implement it?
The input and output checks by cloud, with statuses as the vendor pages showed on 2 Nov 2026.
Capability, Microsoft, AWS, Google
| Capability | Microsoft | AWS | |
|---|---|---|---|
| Input: user prompt and document attacks | Prompt Shields user prompt attack (on by default) and document attack (off by default); Spotlighting preview, models only | Prompt attack filter with input tags; Standard tier adds prompt leakage and code-aware filtering | Model Armor prompt injection and jailbreak detection; floor settings per project |
| Input: tool results | Tool response intervention point (preview); Prompt Shields document attacks on supported tools | Not evaluated by the prompt attack or PII filters; use ApplyGuardrail | Model Armor on Google-managed MCP servers (generally available 22 Apr 2026) |
| Output: text | Harm categories, protected material (text generally available, code preview), PII (preview); groundedness preview retires 30 Apr 2027 | Content filters, sensitive information filters, contextual grounding, Automated Reasoning checks (generally available 6 Aug 2025) | Responsible-AI filters, Sensitive Data Protection, malicious URL detection; streaming sanitization generally available 10 Jul 2026 |
| Output: proposed tool call | Tool call intervention point (preview); Task Adherence (preview); Copilot Studio external threat detection (preview) | AgentCore Policy at the Gateway (generally available 3 Mar 2026) | Model Armor on Agent Gateway (generally available 24 Jun 2026) |
Foundry's guardrail system applies to Foundry Agent Service agents only, not to other agents registered in Foundry Control Plane (Microsoft Learn, 20 Oct 2026).
Frequently asked questions
What is the difference between a user prompt attack and a document attack?
A user prompt attack comes from the person typing: changing the rules, faking a conversation, swapping the persona, encoding the request. A document attack is an instruction hidden in content the agent fetched, such as an email or a tool result. Azure classifies them separately and only the first is on by default.
Should PII be masked on input, on output or both?
It depends on the task. An agent that must read a record to act on it needs the data on input and masking on output; a general assistant can block on both. On Bedrock the two directions are configured separately, and masking never reaches tool arguments, tool results or the invocation logs.
Does an output guardrail check the tool call or only the answer?
Both, once you configure it. Foundry's tool call intervention point, Task Adherence, AgentCore Policy and Model Armor on Agent Gateway all inspect the proposed action. On every platform the tool call check is newer than the text check and, on Foundry, it is off unless the agent guardrail includes it.
How much do input and output guardrails cost in latency?
Each service-side check is a call in the request path, while local rules cost little. Automated Reasoning checks add latency that grows with the policy, and Spotlighting increases token counts. Measure the whole path and keep classifiers off paths a rule already covers.
More in Guardrails
Sources
Each source was read on 2 November 2026. The date shown is the date on the page, or the page’s ms.date for Microsoft Learn; an undated page shows the date it was read.
- Prompt Shields in Azure AI Content Safety
- Guardrails and controls overview in Microsoft Foundry
- What's new in Azure AI Content Safety
- Enable external threat detection and protection for Copilot Studio custom agents (preview)
- Learn about the Microsoft 365 Copilot location for data loss prevention
- How Microsoft defends against indirect prompt injection attacks
- Detect prompt attacks with Amazon Bedrock Guardrails
- Remove PII from conversations by using sensitive information filters
- Amazon Bedrock Guardrails overview
- Automated Reasoning checks in Amazon Bedrock Guardrails
- Policy in Amazon Bedrock AgentCore is now generally available
- Model Armor overview
- Model Armor release notes
- The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections
- ForcedLeak: agent risks exposed in Salesforce Agentforce
Planning an AI program?
Talk to the team that builds it and runs it after launch.
- Built to SOC 2, HIPAA and GDPR standards
- EU AI Act-aligned delivery
- Microsoft Solutions Partner