Govern and secure · Topic 11 of 14
What are the security risks of AI agents, and how do you secure them?
Security threats to AI agents are the ways an attacker turns an agent's access against its owner: hidden instructions in content it reads, abuse of the tools and credentials it holds, poisoned memory or tool supply chain, and data leaks. Many 2025 and 2026 incidents needed no software bug, only an agent with too much reach reading attacker-controlled text.
In this guide: 83 pages
Key takeaways
- The reference lists as of 2 November 2026: the OWASP Top 10 for Agentic Applications for 2026 (released 9 December 2025), the OWASP Top 10 for LLM Applications 2026 (August 2026) and MITRE ATLAS 2026.10 (28 October 2026).
- The 2025 incidents share one shape: untrusted content reached an agent that held private data and could reach out, and the data left through an image fetch, a tool call or a pull request.
- The 2026 CVEs are conventional software bugs in agent platforms: code injection in the AgentCore starter toolkit, a harness that ran tools without model mediation, raw prompts written into CloudWatch spans, an elevation of privilege in Copilot Studio. Patch agent SDKs like any dependency.
- The design rule that holds: never let one agent session combine private data, untrusted content and an outbound channel (Willison's lethal trifecta; Meta's Agents Rule of Two). Least privilege and sandboxing limit what a successful attack can do, which no filter guarantees.
Why are AI agents a new attack surface?
Agents are a new attack surface because they act with real credentials on content they read, and the model cannot distinguish an instruction from its user from an instruction planted in the content it reads. An attacker who can get text in front of an agent can often get its tools to work for them.
For a language model, data and instructions are one stream of tokens. Invariant Labs showed the consequence on 26 May 2025: a malicious issue in a public GitHub repository instructed a user's coding agent, connected through the official GitHub MCP server, to read the user's private repositories and publish the contents in a pull request on the public one. Invariant called it an architectural problem, "not a flaw in the GitHub MCP server code itself".
Credentials make the difference between a bad answer and an incident. In the Supabase demonstration of 8 July 2025, General Analysis showed a support ticket with hidden instructions leading a developer's assistant, running with the service role and so bypassing row-level security, to read a table of integration tokens and post them back into the ticket. In July 2025 Replit's agent ran destructive commands against a production database during a declared code freeze; Replit then separated development and production databases automatically.
Attackers also use agents. Anthropic reported on 13 November 2025 that a group it assesses as Chinese state-sponsored used Claude Code against about 30 targets from mid-September 2025, with AI performing 80 to 90% of the work, and succeeded in a small number of cases.
What are the main security threats to AI agents?
Six threat classes cover almost every documented incident: injected instructions, tool misuse and excessive agency, credential and privilege abuse, memory and context poisoning, a poisoned tool or package supply chain, and data exfiltration. The table pairs each with its framework IDs and the control that addresses it.
Threat classes, framework IDs and the control that addresses each. OWASP mappings follow the research team's judgement.
| Threat | Plain language | Framework IDs | Control that addresses it |
|---|---|---|---|
| Prompt injection (agent goal hijack) | Text the agent reads carries instructions it follows, from the user or from a document, email, web page or tool result | ASI01; LLM01:2026 | Document classifiers, trust tagging, and a design in which a hijacked agent cannot reach private data and an exit together |
| Tool misuse and excessive agency | A legitimate tool is used for harm, or the agent has more tools and permissions than the task needs | ASI02; LLM03:2026 (up from sixth in 2025) | Tool allow list per agent, argument limits in a policy engine, approval on irreversible actions |
| Identity and privilege abuse | Stolen, over-broad or shared credentials let the agent, or whoever holds its token, act beyond scope | ASI03; LLM02 | An identity per agent, short-lived scoped tokens, no human accounts, no token passthrough |
| Memory and context poisoning | Planted instructions or false facts in memory or retrieved context steer later sessions | ASI06; ATLAS AML.T0080 | Write controls and provenance on memory, expiry, a document-attack check on retrieval |
| Agentic supply chain | Malicious or compromised MCP servers, packages, extensions or tool descriptions; rug pulls after approval | ASI04; LLM04:2026; ATLAS AML.T0110, AML.T0109 | Official servers only, pinned tool lists and versions, review on change |
| Data exfiltration | Private data leaves through an image fetch, a link, an email, a tool call or a pull request | LLM02; ATLAS AML.T0086 | Egress allow lists, no untrusted image fetches, approval on outbound actions, DLP |
The other four OWASP agentic items are unexpected code execution (ASI05), insecure inter-agent communication (ASI07), cascading failures (ASI08) and human-agent trust exploitation (ASI09), and the items overlap inside one incident. The sub-page on the OWASP agentic top 10 takes each item in turn.
Which frameworks name the threats: OWASP and MITRE ATLAS?
Three frameworks give security teams a shared vocabulary for agents, and all three changed in 2025 and 2026: the OWASP Top 10 for Agentic Applications, the OWASP Top 10 for LLM Applications 2026 and MITRE ATLAS. Use the OWASP lists for risks in a review and ATLAS for attacker techniques in detection engineering.
The OWASP GenAI Security Project released the OWASP Top 10 for Agentic Applications for 2026 on 9 December 2025 with input from more than 100 contributors. Its items are ASI01 Agent Goal Hijack, ASI02 Tool Misuse, ASI03 Identity and Privilege Abuse, ASI04 Agentic Supply Chain Vulnerabilities, ASI05 Unexpected Code Execution, ASI06 Memory and Context Poisoning, ASI07 Insecure Inter-Agent Communication, ASI08 Cascading Failures, ASI09 Human-Agent Trust Exploitation and ASI10 Rogue Agents. OWASP announced an Agent Control Standard for runtime enforcement on 2 September 2026.
The OWASP Top 10 for LLM Applications 2026 followed in August 2026 (OWASP's resource page says 3 August; its announcement is dated 2 September). No entries were added or removed. Excessive Agency moved from sixth to third, Unbounded Consumption from tenth to sixth, and System Prompt Leakage became Hidden Context Exposure with a wider scope. OWASP's own pages did not list the entries when we checked, so this ordering comes from the Cloud Security Alliance's write-up of 21 October 2026.
MITRE ATLAS added its first agent techniques in version 5.0.0 on 30 September 2025, including AI Agent Context Poisoning, Credentials from AI Agent Configuration and Exfiltration via AI Agent Tool Invocation. It moved to calendar versions in May 2026; the latest, 2026.10 of 28 October 2026, adds Poison AI Agent Learning Environment. For MCP specifically, the OWASP MCP Top 10 (beta) lists tool poisoning, shadow MCP servers and context injection among its ten.
What happened in 2025 and 2026?
Two kinds of incident filled the two years: demonstrations and disclosures in which an agent was steered through content it read, and conventional vulnerabilities in agent platforms with CVE numbers. The table lists the ones with a primary write-up, as the researchers described them.
Documented incidents, in date order. Each description follows the researcher's or vendor's own account.
| Date | Incident | What happened | Fix or lesson |
|---|---|---|---|
| 26 May 2025 | GitHub MCP toxic agent flow (Invariant Labs) | A public issue steered a coding agent into publishing private repository data in a pull request | No server patch possible; one repository per session, runtime guardrails |
| 11 Jun 2025 | EchoLeak, CVE-2025-32711, Microsoft 365 Copilot (Aim Labs) | One crafted email caused zero-click exfiltration past the injection classifier, through auto-fetched images | Fixed by Microsoft server-side; no customer action; no known exploitation |
| 8 Jul 2025 | Supabase MCP demonstration (General Analysis) | A support ticket led an assistant running with the service role to post integration tokens into the ticket | Read-only mode, scope to one project, review tool calls |
| 18 to 21 Jul 2025 | Replit agent deletes a production database | Destructive commands during a code freeze, then a false claim that rollback was impossible | Automatic dev and prod separation, better rollback, planning-only mode |
| Aug 2025 (reported 26 Aug) | Salesloft Drift token theft (Google Threat Intelligence Group) | Stolen OAuth tokens mass-exported Salesforce data at many companies, then were mined for keys | Tokens revoked; Cloudflare rotated 104 API tokens found in the stolen data |
| 20 Aug 2025 | Comet browser injection (Brave) | Hidden text in a Reddit comment made the browser read the user's email and a one-time code | Vendor fixes; Brave found the attack still possible afterwards |
| 25 Sep 2025 | ForcedLeak, Salesforce Agentforce (Noma Labs) | Instructions in a web-to-lead field ran on an ordinary query; data left via an image URL on an expired allow-listed domain | Trusted URL enforcement from 8 Sep 2025 |
| 29 Sep 2025 | postmark-mcp malicious npm package | After 15 clean versions, 1.0.16 copied every email sent through it to an attacker address | Remove the package, check email logs, rotate credentials |
| 30 Sep 2025 | CamoLeak, GitHub Copilot Chat (Legit Security) | A hidden HTML comment in a pull request made Copilot leak private repositories through proxy image URLs | GitHub disabled image rendering in Copilot Chat (14 Aug 2025) |
| 4 Aug 2026 | AgentCore harness, CVE-2026-18830 (AWS bulletin 2026-073) | Caller-supplied tool-use blocks let authenticated users run tools without model mediation | Server-side validation; no customer action |
| 3 Sep 2026 | Copilot Studio, CVE-2026-80098 | Elevation of privilege through improper signature verification; Microsoft 9.3, NVD 10.0 | Fixed by Microsoft; no customer action |
| 6 Oct 2026 | AgentCore Starter Toolkit, CVE-2026-105812 | Unencoded configuration values in generated Python allowed code execution on import | Upgrade to 0.3.14 and re-import agents |
Others belong on the same list: the Amazon Q Developer extension compromise of July 2025 (CVE-2025-8217), the OpenClaw one-click remote code execution of February 2026 (CVE-2026-25253), the Moltbook exposure of about 1.5 million agent API tokens through a client-side database key (Wiz, February 2026), and three more AgentCore CVEs in 2026 (CVE-2026-4269, 12530 and 15737, the last of which wrote raw prompts into CloudWatch spans). Agent platforms are software and need the patch cadence of a web framework.
What is the lethal trifecta, and how do you design around it?
The lethal trifecta is Simon Willison's name (16 June 2025) for the combination that lets an attacker steal data through an agent: access to private data, exposure to untrusted content and the ability to communicate externally. Design around it by making sure no single agent session has all three, and by giving the agent the least privilege and the smallest sandbox that still does the job.
Meta turned the same idea into an operating rule on 31 October 2025. The Agents Rule of Two says an agent must satisfy no more than two of three properties within a session without supervision: processing untrustworthy inputs, having access to sensitive systems or private data, and being able to change state or communicate externally; an agent that needs all three gets a human approval on that last step. The practical move is to split one agent into two: a reader that sees untrusted content and produces a structured summary, and an actor that never sees the raw content.
Least privilege is the control the 2025 write-ups keep returning to. Cloudflare's write-up of the Drift incident (2 September 2025) sets out the controls: audit every vendor application for minimum access, keep administrator accounts away from vendors, rotate integration keys on a schedule, and scope what each third-party tool can reach. The MCP specification (2026-07-28) forbids token passthrough, requires per-client consent in proxies, recommends minimal initial scopes with step-up, and tells servers never to treat possession of a state handle as authentication.
Sandboxing bounds what a successful attack can do. Computer-use and code-execution agents belong in a minimal-privilege virtual machine with no credentials in reach and a domain allow list; Microsoft's documentation says computer use "comes with significant security and privacy risks, including prompt injection attacks". Pin tool lists so a server cannot change a tool description after approval (Anthropic's MCP client version of 15 September 2026 and AgentCore Gateway's static tool schema both do this), and treat provider-supplied tool descriptions as untrusted input, which is Microsoft's guidance for Foundry.
How do Microsoft, AWS and Google implement it?
The threat detection and response controls by cloud, with statuses as the vendor pages showed on 2 Nov 2026. Guardrail products are on the guardrails topic page.
Capability, Microsoft, AWS, Google
| Capability | Microsoft | AWS | |
|---|---|---|---|
| Runtime detection and blocking on tool calls | Defender real-time protection: Agent 365 through Work IQ MCP; Copilot Studio and Foundry agents in preview; local agents through Defender for Endpoint | AgentCore Policy at the Gateway (generally available 3 Mar 2026); Gateway authorization | Model Armor on Agent Gateway (generally available 24 Jun 2026) and on Google-managed MCP servers |
| Injection detection on content | Prompt Shields user prompt and document attacks; Copilot Studio built-in blocking of user and cross-domain injection | Bedrock Guardrails prompt attack filter (skips tool results) | Model Armor prompt injection and jailbreak detection |
| Threat alerts into the SOC | Defender for Cloud threat protection for AI services (generally available); Defender XDR BehaviorInfo table | CloudTrail and CloudWatch; AWS security bulletins for AgentCore CVEs | Security Command Center with Model Armor findings; Google Threat Intelligence Group reporting |
| Agent identity and token scope | Microsoft Entra Agent ID with Conditional Access for agents | AgentCore Identity workload identities and outbound credentials | SPIFFE-based Agent Identity (API in preview since 18 Jun 2026) |
| Sandbox for code and computer use | Foundry computer use tool (limited-access preview model); Copilot Studio computer use (generally available May 2026) | AgentCore Code Interpreter and Browser; 2026 SDK and toolkit CVEs patched | Computer Use sandbox (generally available 9 Sep 2026) with a human-in-the-loop safety constraint |
| Platform CVEs in 2026 | CVE-2026-21520 and CVE-2026-80098 in Copilot Studio, both mitigated by Microsoft with no customer action | CVE-2026-4269, 12530, 15737, 18830 and 105812 in AgentCore; four need customer upgrades, one of them with a re-import, and one was fixed server-side | None found in the research pass; treat as not checked rather than clean |
Defender's coverage of Copilot Studio and Foundry agents was preview on 2 Nov 2026, and the Microsoft cloud CVEs carry "no customer action", which means customers cannot verify their own exposure windows.
When does agent security go wrong, and what should you not do?
Do not give an agent a human's account, a service role or an admin token to save time in a pilot; the Supabase and Replit incidents started that way. Do not treat a vendor's fix for one exit (image rendering, one domain) as a fix for the pattern. Once an agent is live, these are the documented ways security fails.
No patch for the pattern
Invariant said the GitHub MCP issue could not be fixed server-side, and Brave found the Comet attack still possible after Perplexity's fixes. Any agent that combines untrusted input, private data and an outbound channel inherits the risk, whichever vendor built it.
Exits nobody listed
EchoLeak, ForcedLeak and CamoLeak all moved data through image fetches: reference-style markdown, an expired allow-listed domain, pre-signed proxy URLs. ShadowLeak sent data from OpenAI's own infrastructure, which Radware described as invisible to local defenses. Enumerate every way bytes can leave.
Over-broad tokens and shared credentials
Stolen Drift tokens exported Salesforce data at many companies and were mined for AWS keys and Snowflake tokens. Copilot Studio warns that sharing an agent configured with maker-provided credentials lets anyone using it act with the maker's access.
Tool descriptions as code
Tool poisoning hides instructions in a tool description the user never sees; a rug pull changes the description after approval; shadowing lets one server alter how an agent uses another. Annotations such as read-only are hints a dishonest server can misuse.
Fixes that need customer action
CVE-2026-105812 requires upgrading the AgentCore starter toolkit and re-importing agents; Amazon Q Developer 1.84.0 had to be removed from machines. Track each required customer step to completion.
Agents that misreport
The Replit agent said a rollback was impossible when it was not and generated fake records. OWASP calls this human-agent trust exploitation. Verify an agent's account of its own actions from the trace, never from the agent.
What should you set up?
The security controls we put around an agent before it reads anything an outsider can write, in order.
- A threat model per agent using the OWASP agentic list: untrusted inputs, private data, tools, outbound channels and memory, with the trifecta check written down.
- An identity per agent with short-lived, scoped tokens; no human accounts, no service roles that bypass row-level security, no token passthrough between services.
- A tool allow list pinned at approval time, with tool descriptions reviewed on every change and versions pinned for MCP servers, SDKs and packages; official servers only.
- Deterministic exits: egress allow list, no auto-fetched images or links from untrusted content, outbound email and payments behind approval, DLP on what leaves.
- Runtime detection at the tool call on your platform (Defender real-time protection, AgentCore Policy, Model Armor on Agent Gateway) with blocking rules.
- A sandbox for anything that executes code or drives a browser: minimal-privilege VM, no credentials in reach, domain allow list, human confirmation for consequential actions.
- Protected logs: prompts, tool calls and results captured with PII handling, access limited, retention set, and a check that the SDK is not writing raw content into traces.
- A patch runbook for agent platforms and SDKs, subscribed to the vendor bulletins, and an incident playbook with token revocation, integration disconnection and a kill switch per agent.
Enterprise example · Worked example · illustrative
A 3,000-person insurer secures a claims-intake agent that reads outside email
Illustrative scenario. A property and casualty insurer with 3,000 staff on Microsoft 365 E5 with Agent 365, a Copilot Studio agent that reads inbound claims email, extracts the facts into the claims system and replies to the claimant, and a security team that runs Defender XDR.
The threat model was short. The agent read untrusted content (any email from the public), held private data (the claims system and the claimant's history) and could communicate externally (reply to the claimant, update the record): the full trifecta, in one agent, under the maker's own credentials.
The team split it into two agents. The reader agent received each email, ran under its own Microsoft Entra Agent ID with permission to read the mailbox and nothing else, and produced a structured claim summary with no free text carried through. The actor agent took the summary, read and wrote the claims system under a second agent identity, and drafted a reply that a claims handler approved before it was sent. Neither agent held all three properties at once: the reader had untrusted input and no way to act; the actor treated the summary as untrusted, wrote only typed fields and sent nothing without approval.
Runtime controls covered what the split left open. Copilot Studio's built-in injection blocking stayed on, external threat detection was connected to Defender in every environment with the error behaviour set to Block the query, and a Defender custom rule blocked secret exfiltration and email to unlisted domains. The actor agent had no mailbox or email knowledge source at all, and its HTTP connector was limited to the claims API through a data policy.
Testing and patching completed the controls. Thirty poisoned emails, modelled on EchoLeak and ForcedLeak, ran against the reader agent before launch and after every model update; two reached the summary in the first round and none after the summary schema was tightened. The security team subscribed to the MSRC feed and logged CVE-2026-80098 as mitigated by Microsoft with no customer action. In the first quarter the audit log showed 11 blocked tool calls, all from emails with embedded instructions, and no reply sent without a handler's approval.
- Owner
- Claims operations manager; the CISO signs the threat model and the quarterly review
- Identity
- One Microsoft Entra Agent ID per agent with scoped permissions; no maker credentials; Conditional Access blocks high-risk agent sign-ins
- Trifecta
- Reader agent sees untrusted email and cannot act; actor agent treats the summary as untrusted and sends nothing without a handler
- Runtime detection
- Copilot Studio external threat detection to Defender in every environment, fail closed; Defender custom rules block exfiltration and unlisted email domains
- Data protection
- Actor agent has no mailbox or email source; connectors limited by data policy
- Risk tier
- Tier 3 (reads public input, writes a system of record); 30-case poisoned-email test before launch and after every model change
- Approval
- Every outbound reply approved by a claims handler, with the exact text and recipient shown
- Retirement
- Agent identities disabled, connectors removed, environment detection setting kept for the next agent, logs archived under the claims retention schedule
How do you start?
- List every agent that reads content an outsider can write, and mark which also hold private data and can reach out. Those are first.
- Give each of those agents its own identity and remove any human account, service role or admin token it borrows.
- Write the trifecta decision per agent: which leg you removed, or which step a person approves.
- Close the exits: egress allow list, no untrusted image fetches, approval on outbound email and payments.
- Turn on runtime detection at the tool call with blocking rules, and set fail-closed behaviour where the platform offers it.
- Build a poisoned-content test set from the public incident write-ups and run it before launch and after every model change.
- Subscribe to the vendor security bulletins for every agent platform and SDK you run, and rehearse the patch, re-import and removal steps once.
Frequently asked questions
What are the biggest security risks of AI agents?
Injected instructions in content the agent reads, abuse of the tools and credentials it holds, poisoned memory or tool supply chain, and data leaving through an image fetch, email or tool call. OWASP ranks agent goal hijack first and tool misuse second in its agentic top 10.
How do you secure an AI agent?
Give it an identity of its own with scoped tokens, pin its tools, keep untrusted content, private data and outbound channels out of the same session, close every exit with an allow list, put approval on irreversible actions, detect at the tool call, sandbox code and browser use, protect the logs and patch the platform.
Is prompt injection the same as a jailbreak?
No. A jailbreak is a user trying to get a model past its own safety training. Prompt injection is an instruction smuggled into content the model processes, from the user or from a document or tool result, to make the application do something its owner did not intend. Indirect injection through fetched content is the agent-specific case.
Which security framework should an enterprise use for agents?
Use the OWASP Top 10 for Agentic Applications for 2026 as the review checklist, the OWASP LLM Top 10 2026 for the application around the agent, and MITRE ATLAS (2026.10) for attacker techniques in detection rules.
Were the 2025 and 2026 incidents real breaches or demonstrations?
Both. EchoLeak, ForcedLeak and CamoLeak were researcher demonstrations the vendors fixed before any known exploitation; the GitHub MCP flow and the Supabase case demonstrated a design flaw, mitigated by narrower permissions and tool restrictions. The Salesloft Drift token theft, the postmark-mcp package, the Replit deletion and the Moltbook exposure caused real harm. The Copilot Studio and AgentCore CVEs were vulnerabilities the vendors fixed.
Do we need a kill switch for an agent?
Yes. OWASP lists rogue agents as its tenth item, and the practical controls are the ability to disable the agent's identity, deny its tool calls at the gateway or policy engine, revoke its tokens and disconnect its integrations in minutes. Test the switch on a throwaway agent and write down what each step leaves running.
In this topic
Sources
Each source was read on 2 November 2026. The date shown is the date on the page, or the page’s ms.date for Microsoft Learn; an undated page shows the date it was read.
- OWASP Top 10 for Agentic Applications for 2026
- OWASP Top 10 for Agentic Applications: the benchmark for agent security
- OWASP GenAI Security Project releases Top 10 risks and mitigations for agentic AI security
- OWASP GenAI LLM Top 10 2026
- OWASP GenAI Security Project unveils 2026 Top 10 for LLM Applications and new Agent Control Standard
- The OWASP Top 10 for LLM Applications 2026: what changed and why it matters
- OWASP MCP Top 10 (beta)
- MITRE ATLAS data changelog
- GitHub MCP exploited: accessing private repositories via MCP
- MCP security notification: tool poisoning attacks
- CVE-2025-32711 (EchoLeak)
- Supabase MCP can leak your entire SQL database
- AI coding tool Replit wiped a database and called it a catastrophic failure
- AWS security bulletin AWS-2025-015 (Amazon Q Developer extension)
- Widespread data theft targets Salesforce instances via Salesloft Drift
- The impact of the Salesloft Drift breach on Cloudflare and our customers
- Comet AI browser: indirect prompt injection
- ForcedLeak: agent risks exposed in Salesforce Agentforce
- CamoLeak: critical GitHub Copilot vulnerability leaks private source code
- First malicious MCP server found in the wild (postmark-mcp)
- ShadowLeak zero-click flaw leaks Gmail data
- Disrupting the first reported AI-orchestrated cyber espionage campaign
- Exposed Moltbook database reveals millions of API keys
- CVE-2026-25253 (OpenClaw)
- AWS security bulletin 2026-073 (AgentCore harness)
- CVE-2026-15737 (AgentCore Python SDK logging)
- CVE-2026-105812 (AgentCore Starter Toolkit)
- CVE-2026-80098 (Microsoft Copilot Studio)
- The lethal trifecta for AI agents: private data, untrusted content, and external communication
- Agents Rule of Two: a practical approach to AI agent security
- How Microsoft defends against indirect prompt injection attacks
- MCP security best practices (specification 2026-07-28)
- MCP TypeScript SDK security advisories
- Real-time protection for AI agents during runtime
- Computer use tool (Microsoft Foundry Agent Service)
- Computer use in Copilot Studio
- Model Context Protocol tool (Microsoft Foundry Agent Service)
- Policy in Amazon Bedrock AgentCore is now generally available
- Model Armor release notes
Planning an AI program?
Talk to the team that builds it and runs it after launch.
- Built to SOC 2, HIPAA and GDPR standards
- EU AI Act-aligned delivery
- Microsoft Solutions Partner