- Home
- AI agents
- Observability and cost
- Alerting and incident response
Operate · Observability and cost
How do you set up alerting and incident response for AI agents?
Alerting and incident response for AI agents means watching the signals that show an agent misbehaving, such as runaway tool calls, guardrail blocks, falling evaluation scores or unusual write actions, and having a tested plan to contain it, keep the evidence, recover a known-good version and turn the failure into a test.
In this guide: 83 pages
Key takeaways
- Alert on behaviour the model cannot talk its way out of: tool calls and model calls per run, write actions to new destinations, guardrail and injection flags, approval patterns and online evaluation scores. Cost alerts arrive too late to stop a loop.
- Contain first and diagnose second. Disabling a tool or an agent identity takes minutes, and working out why the agent did something can take days.
- Every off switch leaves something running. A Foundry Control Plane stop leaves runs in progress, a block in the Microsoft 365 admin center only affects Copilot Chat for Foundry and SharePoint agents, and a Copilot Studio quarantine still allows the maker’s test pane.
- Preserve evidence before retention runs out. CloudTrail event history keeps 90 days and Google’s _Default log bucket 30 days, so export traces and audit records and set holds as part of containment.
- Two published frameworks cover agent incidents: the OWASP GenAI Incident Response Guide 1.0 (28 Jul 2025) and the CoSAI AI Incident Response Framework V1.0 (27 Oct 2025), which builds on NIST SP 800-61r3.
Which alerts should an agent have?
Start with alerts on the agent’s behaviour, then add quality and identity signals. The table lists the signals we set for production agents, why each one matters and the first thing to do when it fires.
Alert signals for a production agent, with the first response to each. Metric names follow the main branch of the OpenTelemetry GenAI conventions on 2 Nov 2026; the conventions have Development status, so pin the SDK version and retest alerts after upgrades.
| Signal | Fires when | Why it matters | First action |
|---|---|---|---|
| Tool calls and model calls per run | The count jumps well above the baseline for the agent version (gen_ai.invoke_agent.tool_calls and inference_calls) | The earliest sign of a loop or a hijacked plan | Open the live traces; cap or stop the run |
| Write actions to new destinations | The agent emails, posts or writes to a recipient, domain or system it has not used before | Data that leaves the company usually leaves through a write tool | Disable the tool; review the arguments |
| Guardrail and injection flags | Blocks or injection flags rise, for example XPIADetected or JailbreakDetected in Purview audit records | Someone is probing the agent, or a poisoned document sits in its sources | Find the source document or input and quarantine it |
| Error rate and latency | error.type rate or duration per run leaves its normal band | Tool outages and model changes show up here first | Check provider status and recent changes |
| Online evaluation score | The sampled score for a criterion drops below its threshold | Quality drift that no error message reports | Compare with the last regression run; consider rollback |
| Approval pattern | The approval rate nears 100 percent or review time drops to seconds | Automation bias: the human check has become a formality | Review the queue and resample decisions |
| Identity events | The agent identity signs in from a new place, gains grants or is used by another workload | Stolen or over-broad credentials | Disable the identity and rotate secrets |
| Model or configuration change | A model version, prompt or tool changes outside the release pipeline | An unplanned change skipped the regression suite | Roll back to the pinned version |
| Spend per use case | Daily spend passes its budget | Confirms cost damage, too slowly to stop a loop by itself | Check the run-level signals above |
Set baselines per agent version, because a new model or prompt changes the normal number of tool calls. An alert tuned on last month’s version will either fire constantly or stay silent.
How do you rank an agent incident?
Rank by what the agent did and what it can still do; how strange the output looks matters less. The four levels below are the ones we use, and each one sets who gets called and how fast containment must happen.
- Severity 1: data left the company, money moved, or an irreversible action ran without valid approval. Contain at once, call security, legal and the business owner, and start preserving evidence.
- Severity 2: a wrong or unauthorised action that was caught or can be reversed, or a confirmed injection that reached a tool call. Contain within the hour and notify the agent owner and security.
- Severity 3: quality drift, such as an evaluation score below threshold or rising escalations, with no harmful action. Roll back or pause within the working day.
- Severity 4: a cost or performance anomaly with no effect on outputs. Fix it in the next release.
Personal data in a severity 1 or 2 incident can also trigger breach rules such as those in the GDPR, as it would for any other system, so the privacy team belongs in the call tree from the start.
What are the steps when an agent misbehaves?
Detect, contain, preserve, recover and learn, in that order. The CoSAI framework adds agent-specific containment moves: disable tools, tighten authorisation and check least privilege, and for memory, reset it, isolate sessions and purge conversation history.
- Contain. Disable the tool or MCP server involved, then the agent or its identity if the misuse continues, and revoke the OAuth tokens and API keys the agent held. After the Salesloft Drift integration tokens were stolen in August 2025, Cloudflare found 104 of its API tokens in the stolen data and rotated all of them.
- Preserve. Export the traces, audit records and tool server logs for the affected runs, set legal holds, and snapshot the agent version, prompt and memory store before retention defaults delete the evidence.
- Scope. Use trace ids to list every run, user, document and downstream record the agent touched in the window, and check memory for planted instructions that could come back in a later session.
- Eradicate. Fix the cause: narrow the tool permission, remove the poisoned source or patch the SDK. Some fixes need more than an upgrade; the AgentCore Starter Toolkit flaw CVE-2026-105812 required re-importing agents with version 0.3.14 or later.
- Recover. Roll back to the last pinned version through its alias or named endpoint, rerun the regression suite, then restore access in steps.
- Learn. Add the failing runs to the golden dataset as regression tests, update the registry record and risk tier, and tell the affected users and owners what happened.
Keep one runbook per agent with its tools, identities, off switches, owners and evidence stores, and rehearse it with the people on the call tree. A runbook that names the agent without its identity, its tool servers and its alias or endpoint leaves the on-call engineer to find them in the middle of the incident.
What does each platform’s off switch actually stop?
Each switch stops one layer and leaves others running, so a runbook names several. The table lists the switches the vendors documented on 2 Nov 2026 and what each one leaves behind.
Off switches by platform and what they leave running, as documented on 2 Nov 2026.
| Platform and action | What it stops | What it leaves running |
|---|---|---|
| Microsoft 365 admin center: Block agent | Access to the agent across the organisation | For Foundry and SharePoint agents, only availability in Copilot Chat changes |
| Foundry Control Plane: Stop, for hosted or published agents | New work on the agent | Runs already in progress |
| Foundry Control Plane: Block, for custom agents | Incoming requests | Credentials and tools the agent holds elsewhere |
| Copilot Studio: SetAsQuarantined through the Power Platform API, or Block in inventory | Use of the agent by its audience | The maker’s test pane |
| Entra Agent ID: disable the agent identity or blueprint | Token issuance for that identity, reversibly | The objects themselves; disabling a blueprint across tenants can break dependent Microsoft experiences |
| Entra Conditional Access templates that block agent authentication | A tenant-wide, reversible hold on agent sign-ins | Access through credentials outside Entra, such as API keys stored in tools |
| Bedrock Agents Classic: set the alias to REJECT_INVOCATIONS | Invocations through that alias, without IAM changes | Other aliases and direct access to the tools |
| AgentCore: repoint the named endpoint to an earlier version, or deny the tool in AgentCore Policy | The faulty version, or the tool call at the Gateway | Calls that bypass the Gateway |
| Google: block at Agent Gateway and remove IAM bindings | Calls through the Gateway and access granted by those bindings | Calls that bypass the Gateway |
Test every switch on a throwaway agent each quarter, time it, and write down what kept running. Record the result next to the agent in the registry, so the person on call knows which switch to use first.
How do Microsoft, AWS and Google implement it?
The alerting and containment tooling each cloud offered, as the vendor pages showed on 2 Nov 2026.
Capability, Microsoft, AWS, Google
| Capability | Microsoft | AWS | |
|---|---|---|---|
| Alerts on agent metrics | Alerts in the Foundry Agent Monitoring Dashboard (preview); Azure Monitor alerts on Application Insights data | CloudWatch alarms on AgentCore session count, latency, token usage and error rates | Cloud Monitoring; Agent Observability, in preview at the platform launch |
| Failure pattern detection | Continuous evaluation on sampled runs; AI Red Teaming Agent scans (preview) | AgentCore Failure Insights (preview since July 2026) finds recurring failures, including silent ones | Not verified for this page |
| Identity-level stop | Disable the Entra agent identity; Conditional Access templates | Deny the tool in AgentCore Policy; remove IAM permissions | Remove IAM bindings; Agent Gateway policy |
| Rollback | Immutable Foundry agent versions; Power Platform pipelines can redeploy earlier solution versions | AgentCore Runtime named endpoints pinned to an earlier version | Agent Runtime revisions with traffic splitting (preview since 19 May 2026) |
Frequently asked questions
Who should be on call for an AI agent?
The platform team for containment, because it controls identities, the gateway and the off switches, with the agent owner as the second call for diagnosis. Security joins for anything involving data leaving the company or a suspected attack, and the business owner decides when the agent goes back into service.
Do agent incidents have to be reported to a regulator?
Sometimes, under rules that already apply to you. Assess every personal data breach: under the GDPR you notify the supervisory authority unless the breach is unlikely to put people at risk, and you document each breach either way. Financial firms have their own incident regimes, and the EU AI Act adds serious-incident reporting for high-risk systems as those duties start. Map each agent’s tier to its duties when you register it.
How often should we test the off switch?
Each quarter for production agents and after each platform change, on a throwaway agent that mirrors production. Time each switch and record what kept running, because the vendors document gaps, such as a Foundry stop that leaves runs in progress.
What should the first alert be for a new agent?
Tool calls per run against the baseline for that agent version, which shows loops, hijacked plans and broken tools across runs without any content capture, using the gen_ai.invoke_agent.tool_calls metric from the OpenTelemetry conventions. The metric is recorded per invocation, so stop a loop inside a single run with hard limits on tool calls and run duration.
More in Observability and cost
Sources
Each source was read on 2 November 2026. The date shown is the date on the page, or the page’s ms.date for Microsoft Learn; an undated page shows the date it was read.
- Agent actions in the Microsoft 365 admin center
- Manage agents in Foundry Control Plane
- Use the Power Platform API to quarantine agents
- Disable agent identities
- Agent Monitoring Dashboard in Microsoft Foundry
- Audit logs for Copilot and AI applications
- Deploy and use an Amazon Bedrock agent
- AgentCore Runtime versioning and endpoints
- Amazon Bedrock AgentCore release notes
- CVE-2026-105812
- Working with CloudTrail event history
- Semantic conventions for GenAI metrics
- GenAI Incident Response Guide 1.0
- AI Incident Response Framework V1.0
- The impact of the Salesloft Drift breach on Cloudflare and our customers
Planning an AI program?
Talk to the team that builds it and runs it after launch.
- Built to SOC 2, HIPAA and GDPR standards
- EU AI Act-aligned delivery
- Microsoft Solutions Partner