Plan · Topic 01 of 14
What is an AI agent?
An AI agent is software that uses an AI model to reach a goal: it decides the steps, calls tools such as APIs and business systems, checks the result, and hands back to a person when it should. A chatbot answers questions; a copilot helps a person; an agent does part of the work itself, inside limits the company sets.
In this guide: 83 pages
Key takeaways
- An agent is defined by who decides the next step: the model does, at run time, inside limits you set. A chatbot answers, a copilot suggests, and a scripted flow follows a path a developer wrote in advance.
- An enterprise agent is built from the same parts: a model, instructions, tools, knowledge, memory and orchestration, plus four controls outside the model: identity, human approval, guardrails, and traces tied to a registry record.
- Autonomy is a design choice, separate from capability. Pick a level per workflow, from the agent proposing and a person acting to the agent acting and a person monitoring, and raise it only when the evaluation results justify it.
- Agents pay off on judgment-heavy, multi-system work with unstructured inputs and changing rules. For stable, rule-based, high-volume steps a deterministic workflow is cheaper and more predictable; Anthropic and OpenAI both say to start with the simplest thing that works.
- All three clouds converged on the same building blocks in 2026, and the names changed: Microsoft Foundry, Amazon Bedrock AgentCore (Bedrock Agents is now Classic, closed to new customers since 30 Jul 2026) and Gemini Enterprise Agent Platform, formerly Vertex AI.
What does an AI agent do, exactly?
An agent runs a loop. A trigger starts a run; the model reads its instructions and the current context, then either answers or asks for a tool call; the runtime executes the call, adds the result to the context, and the model goes again until an exit condition is met.
The trigger can be a message in Teams, an event such as a new ticket, or a schedule. On each turn the model sees the instructions you wrote, the conversation and tool results so far, and the tools it may call, and replies with text or with a structured tool call: the tool’s name and its arguments as JSON. The model never runs the tool. Your software does, checks the result and hands it back. OpenAI’s guide to building agents describes this run loop and its exit conditions: a final answer, a "final output" tool, a turn or failure limit, an error, or a hand-off to a person.
Anthropic calls the unit at the centre an augmented LLM: a model with retrieval, tools and memory added. In its words, workflows route models and tools "through predefined code paths" while agents "dynamically direct their own processes and tool usage". That one distinction decides how much you can test in advance, how predictable the cost is, and which controls have to sit outside the model.
In 2026 the loop itself became a product. Vendors call the code that runs it a harness: it decides when to call the model, what context to send, runs the tools the model asks for, and repeats. Amazon Bedrock AgentCore’s managed harness went generally available on 17 Jun 2026, Copilot Studio runs every agent on one of three harnesses, and Microsoft Foundry (Copilot harness, preview), Anthropic (Claude Managed Agents, beta since 8 Apr 2026), OpenAI (Agents API, public beta since 10 Sep 2026) and Google (Managed Agents API, preview since 19 May 2026) sell the same thing under their own names.
What is an enterprise AI agent made of?
Six parts inside the agent and four controls around it: a model, instructions, tools, knowledge, memory and orchestration inside; a runtime, an identity, a human approval step and observability tied to a registry record around it. The table says where each part lives on the three clouds.
The parts of an enterprise AI agent and where each one lives, with statuses as the vendor pages showed on 2 Nov 2026.
| Part | Microsoft | AWS | |
|---|---|---|---|
| Model: reasons and picks the next step | Foundry Models catalog and model router | Amazon Bedrock models; Intelligent Prompt Routing | Gemini models and Model Garden |
| Instructions: goal, rules, forbidden actions | Prompt agent definition in Foundry; Copilot Studio instructions | AgentCore harness configuration or your code | ADK or Agent Designer |
| Tools: APIs, connectors, MCP servers, computer use | Foundry Toolboxes (preview); Power Platform connectors | AgentCore Gateway: APIs, Lambda and MCP servers as tools | Agent Gateway; Gemini function calling |
| Knowledge: company content with the user’s permissions | Azure AI Search; Foundry IQ knowledge bases; SharePoint | Bedrock Knowledge Bases; managed knowledge bases on AgentCore | Agent Search and Agent Retrieval, formerly Vertex AI Search and Vector Search |
| Memory: within and across sessions | Foundry memory (preview); Copilot Studio memory | AgentCore Memory (GA) | Agent Platform Memory Bank |
| Orchestration and runtime: the loop and its sessions | Copilot Studio harnesses; Microsoft Agent Framework 1.0; Foundry Agent Service | AgentCore harness and Runtime; Strands Agents | ADK 2.0 on Agent Runtime, formerly Vertex AI Agent Engine |
| Identity: who the agent is, whose rights it uses | Microsoft Entra Agent ID (GA 1 May 2026) | AgentCore Identity (GA 13 Oct 2025) | Agent Identity, SPIFFE-based; API in preview |
| Human approval: a pause on risky steps | Agent flows approvals; Agent Framework tool approval; Foundry MCP tool require_approval | Harness return-of-control tools; Step Functions callbacks | ADK tool confirmation (experimental) |
| Observability and registry: traces, tests, owner, approval state | Foundry tracing and evaluators; Agent 365 registry | AgentCore Observability and Evaluations; AWS Agent Registry | Agent Observability and Agent Registry |
The anatomy sub-page covers each part, including agent skills and the AGENTS.md file for coding agents.
How much autonomy should an AI agent have?
As little as the workflow needs to pay off, raised in steps as the evaluation results earn it. Autonomy is a design choice separate from how capable the model is, and two public frameworks give you a vocabulary for the choice.
The University of Washington paper "Levels of Autonomy for AI Agents" (June 2025) names five levels by the role the user plays: Operator (the user directs, the agent acts), Collaborator (they plan and execute together), Consultant (the agent leads and consults the user), Approver (the user is involved only for risky or pre-specified cases) and Observer (full autonomy, the user monitors). A very capable model can and usually should run at Approver.
AWS’s Agentic AI Security Scoping Matrix (21 Nov 2025) cuts the same ladder into four scopes, from no agency (read-only, fixed steps, humans approve everything) to full agency (the agent starts work from triggers in its environment and runs continuously), and lists the security controls each scope needs. The scope you choose states which controls you have built.
Two triggers for a human step come from OpenAI’s guide: a failure threshold, such as too many retries, and high-risk actions, meaning anything sensitive, irreversible or high-stakes. The mechanics are the same everywhere: a tool is marked as needing approval, the run pauses with the tool name and arguments, a person decides, and the run resumes. OpenAI’s SDK documentation adds a warning worth copying: approval does not guarantee exactly-once execution, so any tool with a side effect needs an idempotency key.
What are AI agents good at, and what are they bad at?
Agents earn their cost on work that needs judgment at each step, reads unstructured inputs and spans several systems. They are a poor choice for stable, rule-based, high-volume steps, for anything that must run exactly once, and for any task you cannot write a test set for.
OpenAI’s guide gives three tests for when an agent beats a scripted solution: complex decision-making, rules that are hard to maintain, and heavy reliance on unstructured data; otherwise "a deterministic solution may suffice". Anthropic’s advice is to find the simplest solution possible and add complexity only when needed. Both tests belong in the scoping conversation before any build.
Publicly reported deployments show the good fit. Commonwealth Bank of Australia’s ChatIT agent in Microsoft Teams takes a description of an IT problem, runs fixes such as clearing a cache or resetting a session, and hands off to a specialist when it cannot resolve the issue; Microsoft reported more than 12,000 automated fixes in six months at about two minutes each, against a typical 17-minute service-desk call (Microsoft Source Asia, 1 Jul 2025; the figures are CommBank’s own). The pattern to copy is a bounded set of actions, a measurable baseline and a person at the end.
Reliability is the first limit. On tau-bench, where an agent must use tools while talking to a simulated customer, the June 2024 paper reported top-scoring function-calling agents succeeding on under half the tasks and passing all eight runs of the same task under a quarter of the time in retail. METR’s May 2026 report found agents did far worse on open-ended tasks than benchmarks suggested, and that on tasks over eight hours at least 16% of successful runs were illegitimate on review. Cost is the second limit: Anthropic measured agents at about four times the tokens of a chat, and multi-agent systems at about fifteen times. Exactly-once execution is the third.
Good fit, poor fit, and the test to apply before building.
| Good fit | Poor fit | The test |
|---|---|---|
| Judgment at each step: triage, exceptions, investigations | A fixed decision tree that already works | Can you draw the flowchart? If yes, build the flowchart |
| Unstructured inputs: emails, documents, screenshots | Clean structured records with known fields | Does the input need reading, or only parsing? |
| Rules that change faster than a developer can maintain them | Rules that must be applied identically every time | Must the same input always give the same output? |
| A low cost per wrong step, or a person approving the costly ones | Irreversible, unattributable or exactly-once actions | How much damage can one bad tool call do? |
The levels-of-autonomy sub-page expands the workflow-or-agent decision; the page on why AI agents fail in production has the sourced failure data.
Is an AI agent the same as agentic AI, a copilot, a chatbot or RPA?
No. Agentic AI is the adjective form of the same idea; a copilot helps a person who stays in charge; a chatbot answers; RPA and workflow automation follow a script written in advance. The test is who decides the next step and who acts in systems.
The lines blur inside products. Microsoft said in October 2024 that "Copilot is how you’ll interact with these agents", Copilot Studio runs deterministic agent flows and goal-driven agents on the same harnesses, and UiPath launched a platform on 30 Apr 2025 to unify agents, robots and people. Gartner, as reported by Reuters on 25 Jun 2025, estimated that only about 130 of the thousands of vendors selling agentic AI were real, and called the rest agent washing. The sub-page on agents vs copilots, chatbots and RPA has the full comparison and a five-question test.
Who decides and who acts, by kind of software.
| Question | Chatbot | Copilot | AI agent | RPA or flow |
|---|---|---|---|---|
| Who decides the steps | Nobody | The person | The model, inside guardrails | The developer, in advance |
| Who acts in systems | Nobody | The person | The agent, through tools | The bot or flow, by script |
| Predictability | High | High | Lower; needs evaluation and limits | Very high; same input, same output |
Which platforms do enterprises build AI agents on?
Usually the platform where their data and identities already live. 2026 was the year of renames and retirements, so check the name and status before you commit. The table lists the main platforms as of 2 Nov 2026.
The main agent platforms, with status as the vendor pages showed on 2 Nov 2026.
| Platform | Vendor | What it is for | Status seen 2 Nov 2026 |
|---|---|---|---|
| Microsoft Copilot Studio | Microsoft | Low-code agents and deterministic agent flows; computer use; publishes to Teams and Microsoft 365 Copilot | Computer use GA May 2026; A2A GA Apr 2026; GitHub Copilot harness in production-ready preview since Jun 2026 |
| Microsoft Foundry Agent Service | Microsoft | Pro-code agents on Azure: prompt agents, hosted agents in your container, Toolboxes, tracing, evaluators | GA May 2025, then named Azure AI Foundry Agent Service, now Microsoft Foundry (renamed at Ignite, 18 Nov 2025); hosted agents GA per the Azure update of 2 Jun 2026 |
| Amazon Bedrock AgentCore | AWS | Modular platform for any framework or model: Runtime, Gateway, Identity, Memory, Policy, Evaluations, Observability, harness, Registry | GA 13 Oct 2025; Policy GA 3 Mar 2026; Evaluations GA 31 Mar 2026; harness GA 17 Jun 2026; Registry GA 31 Aug 2026 |
| Amazon Bedrock Agents Classic | AWS | The original managed agents, renamed Classic | Closed to new customers since 30 Jul 2026; no end-of-life date; AWS points new work to AgentCore |
| Gemini Enterprise Agent Platform | ADK and Agent Designer to build; Agent Runtime; Agent Registry, Agent Identity, Agent Gateway and Model Armor to govern | Vertex AI renamed to this on 22 Apr 2026; Gateway, Registry and Observability GA 18 Jun 2026; Agent Identity API and Managed Agents API in preview | |
| OpenAI agents platform | OpenAI | Agents SDK (your code runs the loop); Agents API (a managed harness); Responses API | Agents API public beta 10 Sep 2026; Assistants API shut down 26 Aug 2026; Agent Builder shuts down 30 Nov 2026 |
| Claude Agent SDK and Claude Managed Agents | Anthropic | SDK on the Claude Code harness; hosted long-running agents with sandboxes and memory | Managed Agents beta since 8 Apr 2026; models also on Bedrock, Google Cloud and Microsoft Foundry |
Agents embedded in business applications are the other route: Salesforce Agentforce 360 (generally available 13 Oct 2025) and ServiceNow AI Agents with AI Control Tower (GA 6 May 2025). The platforms topic compares Copilot Studio with Microsoft Foundry, the three clouds with each other, and build against buy.
How do Microsoft, AWS and Google implement it?
How each cloud lets you build an agent, with statuses as the vendor pages showed on 2 Nov 2026.
Capability, Microsoft, AWS, Google
| Capability | Microsoft | AWS | |
|---|---|---|---|
| Low-code builder for business teams | Copilot Studio; Agent Builder inside Microsoft 365 Copilot (no external actions) | Amazon Quick, an agentic workspace for business users | Agent Designer in Gemini Enterprise (announced 22 Apr 2026) |
| Pro-code service | Microsoft Foundry Agent Service; Microsoft Agent Framework 1.0 (GA 3 Apr 2026) | Amazon Bedrock AgentCore with Strands Agents or any framework | ADK Python 2.0 (GA 19 May 2026) on Agent Runtime |
| Managed harness, the loop as a service | Copilot Studio harnesses; Foundry prompt agents with the Copilot harness (preview) | AgentCore managed harness (GA 17 Jun 2026) | Managed Agents API (preview since 19 May 2026) |
| Policy on tool calls | Foundry agent guardrails at the tool-call point (preview); Entra Conditional Access for agents | AgentCore Policy (GA 3 Mar 2026), Cedar rules on every Gateway call | Agent Gateway (GA 18 Jun 2026) with Model Armor inline |
Google’s release notes carry two labels for Agent Identity, so this page calls the API a preview. The Foundry hosted-agents GA date differs between an Azure update (2 Jun 2026) and Microsoft’s Build recap (early July 2026).
When should you not use an AI agent, and what goes wrong?
Do not build an agent for a step a flowchart already handles, for an action that must run exactly once with no approval, or for a workflow you cannot write a test set for. Once you build one, these are the ways it goes wrong.
Agent washing
A chatbot or a scripted flow sold or built as an agent. The test is who decides the next step; if the answer is a developer or nobody, call it what it is and run it with the cheaper controls.
Too much autonomy for the risk
An agent acting without approval on a process that needed approval on every change. OWASP’s Top 10 for Agentic Applications (9 Dec 2025) lists the results: tool misuse, identity and privilege abuse, cascading failures and rogue agents.
Goal hijack through what the agent reads
Emails, web pages, documents and tool results can carry instructions the model follows (OWASP ASI01). Treat every input channel as untrusted and check tool results as well as the chat box.
Shared or human credentials
An agent running under a person’s account or a shared service principal makes its actions unattributable; Microsoft’s docs say agent identities exist to tell agent operations apart from those of workforce, customer and workload identities.
Rubber-stamp approvals
People approve what the agent proposes without reading it (OWASP ASI09). Gate by risk tier, show the exact tool and arguments, and auto-allow safe reads so the approvals that remain get read.
Benchmarks that overstate, and platforms that move
METR found at least 16% of successful long runs were illegitimate on review, so only your own evaluation set counts. In 2026 alone Amazon Bedrock Agents became Classic, OpenAI shut down the Assistants API and Vertex AI was renamed: keep agent logic in framework code and use MCP, A2A and OpenTelemetry at the edges.
What should you set up before the first agent?
The minimum we install before an agent touches a production system, in the order we install it.
- A written definition of done: the workflow, the inputs, the acceptance criteria and the baseline.
- One autonomy level per workflow, with the AWS scope it maps to and the controls that scope requires.
- An identity of its own for the agent, and a decision per tool whether it acts for the user or as itself.
- Tools with a name, a description and a JSON schema, fewer than twenty active at once, every write marked for approval or a policy check.
- Guardrails outside the model: input checks on every untrusted channel, a policy engine on tool calls, egress limits.
- An evaluation set with the expected tool trajectory per case, run before release and whenever the model, a prompt or a tool changes.
- Tracing of every model call and tool call with a cost per run, and a registry record with an owner, a risk tier, an approval state and a tested kill switch.
Enterprise example · Worked example · illustrative
A 3,500-person industrial distributor gives an order-change agent Level 4 autonomy
Illustrative scenario. An industrial distributor with 3,500 staff, Microsoft 365 E5, Dynamics 365 for orders, and a 40-person customer-service team handling about 1,200 order-change requests a week by email.
The request was simple to state: customers email changes to open orders (quantities, ship dates, addresses), and an agent should read the email, find the order, propose the change and apply it. We split it by risk before anything was built. Reading an order and drafting a reply is reversible. Changing an order carries real cost: a wrong ship date wastes a truck and a wrong quantity wastes the margin on the line.
The agent runs in Copilot Studio at Level 4, Approver. It reads the email and the order on behalf of the signed-in service representative, drafts the change and the customer reply, and pauses. The representative sees the order lines before and after, approves or edits, and the change is written to Dynamics 365 through an agent flow that runs the same way every time. Cancelling an order and changing a price route to a person with no draft. The agent has no cross-session memory, no access to pricing fields and no HTTP connector.
The evaluation set came from 60 real emails from the previous quarter, each paired with the change the team actually made. The acceptance criteria said the agent had to find the right order in 58 of 60 cases, propose the correct change in 54 of 60, and never propose a change to a closed order. The first build found the right order in 51 of 60; the misses were emails quoting a purchase-order number instead of the sales-order number, so the team added a purchase-order lookup tool and the build passed on the second run. Level 4 stays until three months of approval logs show the representatives editing fewer than 5% of proposed changes.
- Owner
- A named service-operations lead; the head of customer service as sponsor
- Autonomy level
- Level 4 Approver; AWS Scope 2, changes only with human approval
- Identity
- Microsoft Entra Agent ID created with the agent; acts on behalf of the signed-in representative
- Risk tier
- Tier 3, writes to a system of record: approval, an evaluation gate and monitoring mandatory
- Tools
- Order lookup, purchase-order lookup and reply draft run freely; order change needs approval; cancel and price change are not tools
- Guardrails
- Built-in prompt injection protection; a data policy that blocks unauthenticated chat and limits HTTP to approved endpoints
- Approval
- The Requests queue before publication; every order change approved with the before-and-after lines shown
- Evaluation gate
- 60 real emails: 58 of 60 orders found, 54 of 60 correct proposals, zero changes to closed orders; rerun on every change
- Monitoring
- Traces of every run; a weekly review of edited approvals; the 5% edit rate decides whether the level moves
- Retirement
- Blocked, then deleted with the 30-day recovery window; identity disabled; evaluation results archived
How do you start?
- Pick one workflow with a measurable baseline, a bounded set of actions and a person at the end of the path.
- Run the two tests: can you draw the flowchart, and does the input need reading rather than parsing? If the flowchart wins, build it.
- Write the acceptance criteria and collect the evaluation set from real cases before anyone opens a builder.
- Choose the autonomy level and the AWS scope, and build on the platform where the data and identities already live, with the agent’s own identity from day one.
- Ship to one team, keep approvals on every write, and review the approval logs before raising the level.
- Register the agent with an owner, a risk tier and a review date, and recheck vendor names and statuses after Ignite and re:Invent in late 2026.
Frequently asked questions
What does an AI agent do, exactly?
It takes a goal, reads its instructions and context, decides whether to answer or call a tool, has the runtime run the tool, reads the result and goes again until it has an answer, hits a limit, fails, or hands the task to a person. Without tools it is a chatbot.
Is ChatGPT an AI agent?
In its chat form it is an assistant: it answers and drafts, and a person acts. OpenAI’s own guide says applications that use a model but do not let it control workflow execution are not agents. OpenAI’s agent products are the Agents SDK, where your code runs the loop, and the Agents API, in public beta since 10 Sep 2026.
What is an AI agent harness?
The code that runs the loop: it decides when to call the model, what context to send, runs the tools the model asks for, and repeats until an exit condition. In 2026 vendors sell it as a managed service, from the AgentCore managed harness (GA 17 Jun 2026) to Claude Managed Agents (beta) and the OpenAI Agents API (public beta).
What are the types of AI agents?
Three classifications are useful: by autonomy, the five levels from Operator to Observer; by what they do, answer, assist or act; and by interface, chat, event-triggered, or computer-use agents that drive a screen. The textbook taxonomy (simple reflex, model-based, goal-based, utility-based and learning agents) describes decision logic and does not map onto products.
How do AI agents work in Microsoft 365 Copilot?
Microsoft 365 Copilot is the assistant people talk to; agents extend it. Agent Builder creates declarative agents from your knowledge sources inside Copilot, without external actions. Copilot Studio adds actions through connectors, MCP servers, agent flows and computer use, publishes the agent to Teams and Copilot, and since July 2026 gives every new agent a Microsoft Entra Agent ID.
Does an AI agent need its own identity?
Yes. An agent under a person’s account or a shared service principal has unattributable actions and breaks the controls that depend on identity. Microsoft Entra Agent ID, AgentCore Identity and Google Agent Identity exist for this.
In this topic
Sources
Each source was read on 2 November 2026. The date shown is the date on the page, or the page’s ms.date for Microsoft Learn; an undated page shows the date it was read.
- Building effective agents
- A practical guide to building agents (PDF)
- What are agent identities? (Microsoft Entra Agent ID)
- Harnesses in Copilot Studio
- Agent flows overview (Copilot Studio)
- What is Microsoft Foundry Agent Service?
- Foundry hosted agents now generally available (Azure update AZ563596)
- Microsoft Agent Framework 1.0
- Tool approval (Microsoft Agent Framework)
- Model Context Protocol tool (Microsoft Foundry Agent Service)
- Guardrails and controls in Microsoft Foundry
- Conditional Access for agents (Microsoft Entra)
- Agent requests
- Agent actions in the Microsoft 365 admin center
- Configure data policies for agents (Copilot Studio)
- Microsoft Agent 365 overview
- What’s new in Copilot Studio
- Build agents with Agent Builder in Microsoft 365 Copilot
- New autonomous agents scale your team like never before
- Amazon Bedrock Agents Classic maintenance mode
- Amazon Bedrock AgentCore is now generally available
- Top announcements of the AWS Summit in New York, 2026
- Policy for Amazon Bedrock AgentCore generally available
- AgentCore Evaluations generally available
- AWS Agent Registry now generally available
- Tool confirmation (Google ADK)
- Use Amazon Bedrock Intelligent Prompt Routing for cost and latency benefits
- Amazon Quick
- Gemini Enterprise Agent Platform release notes
- Welcome to Google Cloud Next ’26
- Levels of Autonomy for AI Agents (arXiv 2506.12469)
- The agentic AI security scoping matrix
- OWASP Top 10 for Agentic Applications for 2026
- Frontier Risk Report, February to March 2026
- tau-bench: a benchmark for tool-agent-user interaction in real-world domains (arXiv 2406.12045)
- How we built our multi-agent research system
- CommBank’s ChatIT helps employees fix IT issues in seconds using generative AI
- Over 40% of agentic AI projects will be scrapped by 2027, Gartner says
- UiPath launches the first enterprise-grade platform for agentic automation
- Human in the loop (OpenAI Agents SDK)
- OpenAI API changelog
- OpenAI API deprecations
- Claude Platform release notes
- Agentforce 360 and the agentic enterprise
- ServiceNow’s latest platform release adds to thousands of AI agents (Yokohama)
- ServiceNow launches AI Control Tower
Planning an AI program?
Talk to the team that builds it and runs it after launch.
- Built to SOC 2, HIPAA and GDPR standards
- EU AI Act-aligned delivery
- Microsoft Solutions Partner