Build · Topic 06 of 14
What is AI agent orchestration, and when do you need a multi-agent system?
AI agent orchestration is the control logic that decides what an agent system does next: which model call, tool or agent runs, in what order, and what happens when a step fails or needs a person. A workflow follows a path a developer wrote; an agent lets the model choose; a multi-agent system splits work across several agents.
In this guide: 83 pages
Key takeaways
- Start with one agent. Anthropic, OpenAI and Cognition give the same advice: the measured gains from splitting work across agents are often small, and token use is about 15 times a chat.
- The split that matters is deterministic versus model-driven. Put the steps you can specify in a workflow and let the model choose only where the input varies.
- Durable execution lets an agent survive a crash, a deploy or a five-day wait for approval without starting over or paying for idle compute.
- Every vendor sells a managed harness in 2026: AgentCore harness (generally available 17 Jun 2026), the Copilot Studio harnesses, Claude Managed Agents (beta), the OpenAI Agents API (public beta) and the Google Managed Agents API (preview).
- Keep the loop in your own code and open protocols at the edges. In 18 months AutoGen was archived, Semantic Kernel became Microsoft Agent Framework, the OpenAI Assistants API shut down and Bedrock Agents Classic closed to new customers.
Why does orchestration matter more than the model you pick?
Orchestration matters more because, in our experience, the failures that reach production are failures of control, cost and recovery, and the model is the same one everyone else can buy. A 2025 study found multi-agent failures cluster in system design, inter-agent misalignment and task verification.
Anthropic measured that an agent uses about four times the tokens of a chat and a multi-agent system about 15 times, and that token usage alone explained 80% of performance variance on the BrowseComp benchmark (13 Jun 2025). Five workers answering a question one agent could have answered is a bill with no benefit.
The MAST study of 17 Mar 2025 ("Why Do Multi-Agent LLM Systems Fail?") catalogued 14 failure modes in three categories and found that gains on popular benchmarks are often minimal. Cognition said the same on 12 Jun 2025: share context and full agent traces, because actions carry implicit decisions and conflicting decisions carry bad results. Agents are also stateful, so errors compound, which is why Anthropic built resume-from-failure with checkpoints and rainbow deployments.
Then there is churn. Within 18 months AutoGen was archived and Semantic Kernel became Microsoft Agent Framework, OpenAI shut down the Assistants API (26 Aug 2026) and set Agent Builder to close on 30 Nov 2026, and Amazon Bedrock Agents Classic closed to new customers (30 Jul 2026). Orchestration logic in your own code survives this; logic in a vendor visual builder may have to be rebuilt.
How does agent orchestration work?
An orchestrator runs a loop: gather context, decide the next step, call a model, a tool or another agent, check the result, and repeat until the goal is met or a person is needed. Products differ in who decides the next step and in what happens when a step fails.
Anthropic's distinction from 19 Dec 2024 is the one most vendors use. Workflows are "systems where LLMs and tools are orchestrated through predefined code paths". Agents are "systems where LLMs dynamically direct their own processes and tool usage". The same note names five workflow patterns (prompt chaining, routing, parallelization, orchestrator-workers and evaluator-optimizer) and advises "finding the simplest solution possible, and only increasing complexity when needed". The page now carries a banner saying the tooling has changed; the advice has held.
In practice an enterprise agent is both. The outer process is a workflow you can draw: receive, classify, fetch the records, draft, check, act, notify. Inside two or three of those steps the model decides, because the input varies too much for rules. Copilot Studio makes the split visible by running everything on one of three harnesses (Microsoft Learn, 19 Oct 2026): a standard harness for rule-based agents and agent flows, a GitHub Copilot harness that "can take a goal, break it into steps, call the right tools" and "adjust when a step fails", and a Customer Conversation harness of specialized sub-agents, each with its own escalation path to a person.
At the edges sit the protocols: MCP (the Model Context Protocol, specification 2026-07-28) for tools and A2A (the agent-to-agent protocol, version 1.0.0 since 12 Mar 2026) for other agents. The loop in between is where the frameworks and harnesses below differ.
Which multi-agent patterns exist, and when does each fit?
Seven patterns cover what the frameworks ship, and the names differ by vendor. The table gives the mechanism, the work each fits and where it ships as of 2 Nov 2026.
Orchestration patterns, the work each fits and where it ships.
| Pattern | How it works | Fits when | Where it ships |
|---|---|---|---|
| Single agent | One model loop with tools; the model picks the next tool call | One domain, one context, a task a person could brief in a paragraph | Every framework and harness |
| Supervisor (orchestrator-workers, manager) | A lead agent plans, spawns workers in parallel and merges results | Broad tasks that split into independent pieces, such as research across sources | Anthropic Research; agents-as-tools in the OpenAI Agents SDK and Strands; Magentic-One in Agent Framework |
| Handoffs | Control and the conversation pass to another agent | A conversation that changes domain mid-way, such as triage then billing | OpenAI Agents SDK; Agent Framework; Strands handoff_to_user (to a person) |
| Group chat | Agents share one conversation; a manager picks who speaks next | Debate or review between specialists on one artefact | Agent Framework (inherited from AutoGen, which is now archived); a Foundry workflow template (preview) |
| Swarm | Agents coordinate through shared memory without a controller | Exploratory work where the path cannot be specified; rarely a first choice | Strands swarm |
| Graph or workflow | Developers define nodes and edges; the model decides only at marked nodes | Any process with a known shape and a few judgment steps | LangGraph; Agent Framework workflows; Strands graph; Google ADK 2.0; LlamaIndex Workflows |
| Remote agents over A2A | Separate services publish an agent card and delegate tasks over A2A | Agents owned by different teams or vendors, separately deployed | Foundry (A2A 1.0 GA); Copilot Studio (GA Apr 2026); Google Agent Registry (A2A v1); Strands; ADK |
Strands puts the choice in one sentence: "Use Graph when you want to define the flow, Swarm when agents should coordinate autonomously, and Workflow when the process is fixed." OpenAI's guide names two shapes, a manager that calls specialists as tools and a decentralized shape with handoffs, and says to get as far as you can with one agent first.
Workflow vs agent vs multi-agent: which do you need?
You need a workflow when you can write the steps down, an agent when the input varies too much to script, and a multi-agent system only when the work splits into pieces that can run without each other's context. In our experience an enterprise process is usually a workflow with two or three agent steps inside it.
Three ways to control the work, compared.
| Question | Deterministic workflow | Single agent | Multi-agent system |
|---|---|---|---|
| Who decides the next step | The developer, in code or a visual flow | The model, within its tools | A lead agent, or each agent on handoff |
| Same input, same path | Yes | No; the path varies with the model output | No, and the variance grows with the agent count |
| Token use relative to a chat | Model calls only where needed | About 4 times, in Anthropic's system (13 Jun 2025) | About 15 times, in Anthropic's system (13 Jun 2025) |
| How you test it | Unit tests per step | An evaluation set of real tasks | Evaluation per agent plus end to end |
| Fits | Approvals, document pipelines, scheduled jobs | One domain with varied input: triage, research, drafting | Broad parallel work: research across sources, content at volume |
| Avoid when | The steps cannot be listed | The task needs five tools from five domains | The pieces share context; coding is the usual example |
A test we apply before adding a second agent: can the lead brief the worker in writing, and will the worker's output make sense without the lead's conversation? If either answer is no, one agent with context compression will usually do better, which is Cognition's conclusion too.
What is durable execution, and why do long-running agents need it?
Durable execution records each completed step of a run so that a crash, a redeploy or a multi-day wait for a person resumes from the last step instead of starting over, without holding compute while it waits. Long-running agents need it because a task that takes hours outlives any single process.
The mechanics are the same across engines. The orchestrator writes each finished model, tool or agent call to a durable log, replays the log after a restart, skips the finished steps, and suspends the run at a timer, an approval or an external event. Orchestration code must therefore be deterministic: every non-deterministic call is wrapped as a recorded step, which is why Microsoft Agent Framework requires context.GetAgent() or app.get_agent() so that agent calls are "properly tracked and checkpointed". Tool calls that are not idempotent, such as payments and emails, can repeat under an at-least-once engine, so each needs an idempotency key.
The engines, as of 2 Nov 2026. Temporal wraps each agent call as an Activity. LangGraph compiles a graph with a checkpointer and a thread_id, which also drives its approval interrupts. Agent Framework 1.0 checkpoints every pattern, and its Durable Extension (preview at 1.0) lets orchestrations "wait for days or even weeks" for a person; on the Azure Functions Flex Consumption plan a 24-hour approval wait pays for a few seconds of execution (Microsoft Learn, ms.date 31 Aug 2026). On AWS, Lambda durable functions (2 Dec 2025) suspend for up to one year at no charge while waiting; Step Functions Standard workflows run up to one year with exactly-once semantics and the .waitForTaskToken callback, while Express workflows stop at five minutes and asynchronous Express is at-least-once. The AgentCore harness drops into Step Functions through an InvokeHarness state.
Long-running agents add the context window. Anthropic's note of 26 Nov 2025: "The core challenge of long-running agents is that they must work in discrete sessions, and each new session begins with no memory of what came before." The fixes are compaction of older context, external memory such as files, subagents that keep side work out of the main context, and durable sessions in a sandbox. Managed runtimes now host this: AgentCore Runtime sessions run up to 8 hours on microVMs or 14 days on Instances, Claude Managed Agents keeps a session log outside the harness so a crashed harness can wake and resume (8 Apr 2026), and the OpenAI Agents API (public beta 10 Sep 2026) saves an agent's progress for long-running tasks.
Which managed harnesses and frameworks exist, and what is their status?
Every major vendor now runs the agent loop for you as a managed harness, and the open-source frameworks reached 1.0 during 2025 and 2026. The table gives the status each vendor page showed on 2 Nov 2026.
Managed harnesses (the agent loop as a service) and orchestration frameworks, with status as seen 2 Nov 2026.
| Product | Vendor | What it gives you | Status as seen 2 Nov 2026 |
|---|---|---|---|
| AgentCore harness | AWS | Declare model, tools, skills and instructions; AWS runs compute, memory, identity and observability; built on Strands | GA announced 17 Jun 2026 (AWS Summit New York); the release notes file harness GA under July 2026 |
| Copilot Studio harnesses | Microsoft | Standard (rule-based agents, flows); GitHub Copilot (goal to steps, tools, files); Customer Conversation (sub-agents, escalation) | In use; GitHub Copilot harness experience a production-ready preview in Jun 2026 |
| Foundry prompt and hosted agents | Microsoft | Config-only agents with an optional Copilot harness; hosted agents take your container | Copilot harness preview; hosted agents GA per an Azure update of 2 Jun 2026; the same-day Build recap said early July |
| OpenAI Agents API | OpenAI | Managed Codex harness; orchestration, compaction and recovery handled by OpenAI | Public beta 10 Sep 2026; visual Agent Builder shuts down 30 Nov 2026 |
| Google Managed Agents API | Managed agents in Gemini Enterprise Agent Platform | Preview 19 May 2026 | |
| Microsoft Agent Framework | Microsoft, open source | Graph workflows; handoff, group chat, Magentic-One; checkpointing; Durable Extension | 1.0 GA 3 Apr 2026; replaces Semantic Kernel and AutoGen, whose repository is now archived |
| LangGraph and LangChain | LangChain, open source | Graph of nodes and edges; checkpointers; approval middleware | 1.0 on 22 Oct 2025 |
| Strands Agents | AWS, open source | Agents-as-tools, swarm, graph, workflow, A2A | 1.0 on 15 Jul 2025; powers the AgentCore harness |
| Google ADK | Google, open source | Sequential, Parallel, Loop agents; LLM transfer; 2.0 graph workflows; A2A | Python 2.0 GA 19 May 2026, breaking changes to the session schema |
| CrewAI | CrewAI Inc., open source | Crews (role-based teams) inside Flows | Version not verified on 2 Nov 2026 |
The frameworks are converging on one shape, a graph with checkpoints, approval interrupts and A2A at the edge, so the choice is mostly which cloud hosts it and which language your team writes. A managed harness is a different trade: you give up the loop's code for operations you no longer run, which suits single agents with standard tools.
How do Microsoft, AWS and Google implement it?
The same capability on each cloud, with the status the vendor pages showed on 2 Nov 2026.
Capability, Microsoft, AWS, Google
| Capability | Microsoft | AWS | |
|---|---|---|---|
| Managed harness | Copilot Studio harnesses; Foundry Copilot harness (preview) | AgentCore harness (GA 17 Jun 2026), built on Strands | Managed Agents API (preview 19 May 2026) |
| Deterministic workflow | Copilot Studio agent flows; Agent Framework workflows; Logic Apps | Step Functions Standard (up to one year, exactly-once); Lambda durable functions (2 Dec 2025) | ADK Sequential, Parallel and Loop agents; ADK 2.0 graph workflows |
| Multi-agent in the product | Foundry multi-agent workflows (public preview since 25 Nov 2025); Copilot Studio connected agents | Strands agents-as-tools, swarm and graph on AgentCore Runtime; the harness covers multi-agent only partly | ADK sub-agents and LLM transfer; agent-to-agent orchestration (announced 22 Apr 2026) |
| Durable execution | Agent Framework checkpointing; Durable Extension (preview at 1.0) | Lambda durable functions; Step Functions callbacks; Runtime sessions up to 8 hours or 14 days | Agent Runtime sessions and Memory Bank; long-running agents in Gemini Enterprise (announced 22 Apr 2026) |
| Human pause point | Agent Framework approval-required tools; Copilot Studio request for information | Step Functions .waitForTaskToken; Lambda durable functions callbacks; Strands handoff_to_user | ADK graph workflows with human input; ADK tool confirmation (experimental) |
| Agent-to-agent | Foundry A2A 1.0 GA (0.3 in preview); Copilot Studio A2A GA Apr 2026 | Strands and AgentCore support A2A; the documented path still pins the 0.3 SDK | Agent Registry with A2A v1 (GA 18 Jun 2026); ADK A2A |
Foundry multi-agent workflows were public preview when announced on 25 Nov 2025, and we could not confirm a later change of status on 2 Nov 2026.
When should you not build a multi-agent system, and what goes wrong?
Do not split a task across agents when the pieces share context, when a person could brief the whole job in one paragraph, or when the budget per task is small. These are the ways orchestration fails in production.
Token cost nobody budgeted
Anthropic measured multi-agent systems at about 15 times the tokens of a chat. A supervisor that fans out five workers for every request turns a cheap task into an expensive one, and spend scales with autonomy and loops rather than with users.
Fragmented context and conflicting decisions
Two workers that never see each other's traces make incompatible choices. Cognition's example was a game where one subagent drew a background in one style and another a mismatched bird; its fix was one agent with context compression.
Replays that repeat side effects
Durable engines replay the log, and a tool call outside the engine's wrapper runs again. LangGraph restarts the whole node on resume, asynchronous Step Functions Express is at-least-once, and OpenAI warns that an approval does not guarantee exactly-once execution. Every write needs an idempotency key.
Paying to wait
An agent that holds a container open while a manager considers an approval for two days is an expensive idle process. Suspend the run instead: Lambda durable functions charge nothing while waiting, and Agent Framework on Flex Consumption pays seconds for a 24-hour wait.
Security that spreads with the agent count
Every worker with tools is another place for prompt injection and over-broad permissions. Handoffs and agent-as-tool calls need the same guardrails and the same scoped identity as a user-facing call, and a worker should never inherit the supervisor's full rights.
What should you set up?
The controls that make an orchestrated agent operable, in the order we install them.
- A written process map with the deterministic steps marked and the two or three model-driven steps circled, agreed before any framework is chosen.
- One framework and one durable engine per cloud, with the loop in your code: Agent Framework with Durable Task on Azure; Strands or LangGraph with Step Functions or Lambda durable functions on AWS; ADK on Agent Runtime on Google Cloud.
- Every model call and tool call wrapped as a recorded step, an idempotency key on every write, and approvals, timers and external events as durable pause points with a timeout and a default of deny.
- A hard cap per run on steps, tokens, time and worker fan-out, enforced by the orchestrator rather than by the prompt; an identity per agent, workers included, with narrower rights than the supervisor.
- Traces per step with the agent, version and session on every span, cost per task on a dashboard, and an evaluation set of real tasks that runs before each release and after each model change.
- Releases with versions pinned, in-flight runs drained before an old version retires, and a tested stop per platform (a Foundry stop does not terminate existing runs).
Enterprise example · Worked example · illustrative
A specialty insurer runs claims intake as a workflow with three agent steps inside it
Illustrative scenario. A specialty insurer with 2,500 staff, a Microsoft 365 and Azure tenant, a policy administration system with an API, and a claims team that receives about 140 first-notice-of-loss emails a day.
The first design was one agent with eleven tools. In evaluation it handled 71 of 120 historical claims correctly, and the failures were untraceable because every decision lived in one long transcript. The team redrew the process as a workflow in Microsoft Agent Framework: receive, classify, extract, look up, check coverage, propose reserve, approve, create claim, notify. Six of the nine steps are code.
Three steps are model-driven: an extraction agent reads the email and attachments into a claim record, a coverage agent reads the policy and returns yes, no or unclear with the clause it relied on, and a reserve agent proposes an amount from similar closed claims. The workflow is the supervisor, so the agents never talk to each other; each gets a written brief from the workflow state, and a verification step compares their outputs before the approval gate.
The run executes on the Durable Extension, so a crash during extraction resumes at extraction, and the approval wait suspends the run for up to five working days without a container. The create-claim and set-reserve tools require approval: an adjuster below a threshold the claims director set, a second person above it. Each tool call carries the claim number as its idempotency key. After four months, 58 of the 140 daily claims reached the adjuster with every field filled and a proposed reserve, and the one incident, a model upgrade that changed how handwritten dates were read, was caught by the 300-claim evaluation run before release.
- Owner
- Claims operations lead as owner, head of claims as sponsor; each agent a separate registry record under the same workflow
- Identity
- One Entra agent identity blueprint, one agent identity per agent; the coverage agent reads the policy system on behalf of the adjuster; the reserve tool runs under the workflow identity with rights to one API
- Orchestration
- Agent Framework workflow, six deterministic steps and three model-driven steps; no agent-to-agent calls
- Durable execution
- Durable Extension on Azure Functions; every call a recorded step; idempotency key per claim
- Risk tier
- Tier 3 because it writes reserves and creates claims
- Approval
- Create-claim and set-reserve tools require approval; adjuster below the threshold, second approver above it; unclear coverage routed to a person first
- Evaluation
- 300-claim set run before each release and after each model change; release blocked below the agreed pass rate
- Monitoring
- Traces per step in Application Insights; cost per claim weekly; alerts on retries and approval timeouts
- Retirement
- Versions pinned; in-flight runs drained before an old version stops; identities disabled and traces archived
How do you start?
- Draw the process as it runs today and mark each step as code or judgment. If there are more than three judgment steps, we treat it as two processes.
- Build the deterministic workflow first with stub agents, and run it end to end on ten real cases.
- Replace one stub with a single agent, give it an evaluation set of real cases, and measure before adding tools.
- Add durable execution before the first approval gate goes in, and prove that a crash and a redeploy resume correctly.
- Add a second agent only when a piece of work can be briefed in writing and completed without the first agent's context, and review the harness table after Microsoft Ignite and AWS re:Invent in late 2026.
Frequently asked questions
What is the difference between an AI workflow and an AI agent?
A workflow follows a path a developer wrote in advance; the model fills in steps but does not choose them. An agent lets the model decide its own next step and tool. Anthropic's definitions from 19 Dec 2024 are the ones most vendors use.
Can Copilot Studio do multi-agent orchestration?
Yes. Connected agents let one agent call others, A2A has been generally available in Copilot Studio since April 2026, and the Customer Conversation harness runs specialized sub-agents with their own escalation paths. For code-level control, Foundry multi-agent workflows (public preview since 25 Nov 2025) and Microsoft Agent Framework are the pro-code options.
What are examples of multi-agent orchestration frameworks?
Microsoft Agent Framework (1.0 generally available 3 Apr 2026), LangGraph (1.0 on 22 Oct 2025), Strands Agents (1.0 on 15 Jul 2025), Google ADK (Python 2.0 on 19 May 2026), the OpenAI Agents SDK, CrewAI and LlamaIndex Workflows. AutoGen is archived and Semantic Kernel became Microsoft Agent Framework, so neither is a starting point for new work.
Is AutoGen still supported now that Microsoft Agent Framework replaces it?
No. The microsoft/autogen repository on GitHub reads "Project archived and no longer maintained" and names Agent Framework as the successor, and the Semantic Kernel repository says it is now Microsoft Agent Framework. The group chat and Magentic-One patterns live on in Agent Framework 1.0.
What is durable execution, and do we need it for every agent?
Durable execution records each completed step so a run resumes after a crash, a redeploy or a long wait instead of restarting, and suspends without compute while it waits. A chat assistant that answers in one turn does not need it. Any agent that waits for a person, runs for minutes or writes to a system of record does.
What are the best practices for multi-agent orchestration?
Start with one agent and a measured evaluation set. Split only work that can be briefed in writing and completed without shared context. Share full traces between agents, verify before writing to systems of record, cap steps and tokens per run, make every step durable and idempotent, and give each agent its own identity.
In this topic
Sources
Each source was read on 2 November 2026. The date shown is the date on the page, or the page’s ms.date for Microsoft Learn; an undated page shows the date it was read.
- Building effective agents
- How we built our multi-agent research system
- Effective harnesses for long-running agents
- Scaling Managed Agents: decoupling the brain from the hands
- Claude platform release notes
- Don't Build Multi-Agents
- Why Do Multi-Agent LLM Systems Fail? (MAST)
- A practical guide to building agents (PDF)
- OpenAI API changelog
- OpenAI API deprecations
- Agents guide (Agents API)
- OpenAI Agents SDK documentation
- LangChain and LangGraph 1.0
- LangGraph durable execution
- Introducing Microsoft Agent Framework
- Microsoft Agent Framework version 1.0
- Durable Extension (Microsoft Agent Framework)
- Introducing multi-agent workflows in Foundry Agent Service
- Harnesses in Copilot Studio
- What's new in Copilot Studio
- Foundry Agent Service overview
- microsoft/autogen repository README (archived)
- Introducing Strands Agents 1.0
- Strands multi-agent patterns
- Build multi-step applications and AI workflows with AWS Lambda durable functions
- Choosing workflow type in Step Functions
- AgentCore harness
- AgentCore Runtime
- Top announcements of the AWS Summit in New York 2026
- Amazon Bedrock Agents Classic maintenance mode
- Agent Development Kit: easy to build multi-agent applications
- Gemini Enterprise Agent Platform release notes
- Welcome to Google Cloud Next '26
- Announcing the OpenAI Agents SDK integration
- A2A protocol specification, version 1.0.0
Planning an AI program?
Talk to the team that builds it and runs it after launch.
- Built to SOC 2, HIPAA and GDPR standards
- EU AI Act-aligned delivery
- Microsoft Solutions Partner